Sep 16 14:04:27.879413 ip-10-0-137-34 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 14:04:28.240087 ip-10-0-137-34 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:04:28.240087 ip-10-0-137-34 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 14:04:28.240087 ip-10-0-137-34 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:04:28.240087 ip-10-0-137-34 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 14:04:28.240087 ip-10-0-137-34 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:04:28.241085 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.240995    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 14:04:28.244541 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244525    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:28.244541 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244541    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244547    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244551    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244553    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244556    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244559    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244562    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244564    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244567    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244570    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244574    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244578    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244581    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244584    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244587    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244590    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244593    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244596    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244599    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244601    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:28.244604 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244604    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244606    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244610    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244613    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244619    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244622    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244625    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244628    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244631    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244633    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244636    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244638    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244641    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244643    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244646    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244648    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244651    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244653    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244655    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244658    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:28.245091 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244660    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244663    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244665    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244668    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244671    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244674    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244676    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244679    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244681    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244684    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244686    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244688    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244691    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244693    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244697    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244700    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244703    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244705    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244708    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244710    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:28.245636 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244713    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244715    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244718    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244721    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244723    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244725    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244728    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244730    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244733    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244735    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244738    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244742    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244745    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244749    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244752    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244755    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244758    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244760    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244763    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244765    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:28.246131 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244768    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244771    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244773    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244776    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244779    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244781    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244784    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244786    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244788    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244791    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244793    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244797    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244801    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244804    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244806    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244809    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244811    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.244813    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246209    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:28.246619 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246217    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246221    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246224    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246227    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246230    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246233    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246236    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246238    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246242    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246244    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246247    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246249    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246252    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246254    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246257    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246260    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246262    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246265    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246269    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246271    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:28.247069 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246274    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246277    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246279    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246282    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246284    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246286    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246289    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246292    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246295    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246297    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246300    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246302    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246304    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246307    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246309    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246312    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246314    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246317    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246319    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246321    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:28.247567 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246324    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246327    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246330    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246332    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246335    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246340    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246342    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246344    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246347    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246349    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246352    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246355    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246357    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246360    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246362    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246365    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246367    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246369    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246372    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246375    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:28.248084 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246377    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246382    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246385    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246388    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246391    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246393    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246397    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246399    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246402    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246405    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246407    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246410    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246413    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246417    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246423    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246427    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246430    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246434    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246437    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:28.248577 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246440    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246443    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246445    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246449    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246452    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246454    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246457    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246459    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246462    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246464    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246467    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246469    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246472    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246474    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246478    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246481    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246484    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246488    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246491    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:28.249033 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246883    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246888    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246891    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246894    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246897    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246900    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246902    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246905    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246908    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246911    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246914    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246918    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246922    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246924    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246927    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246930    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246933    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246935    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246938    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246942    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:28.249509 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246944    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246947    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246950    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246952    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246955    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246958    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246960    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246963    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246965    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246968    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246970    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246973    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246977    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246980    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246982    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246984    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246987    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246989    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246992    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:28.249999 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246994    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.246997    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247000    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247003    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247005    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247008    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247010    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247013    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247015    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247018    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247021    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247023    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247026    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247028    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247030    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247033    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247035    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247038    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247040    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247043    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:28.250474 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247045    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247048    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247050    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247052    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247055    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247057    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247060    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247062    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247065    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247067    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247069    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247072    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247075    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247077    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247080    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247088    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247091    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247094    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247096    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:28.250953 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247099    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247101    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247116    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247119    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247122    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247124    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247127    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247129    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247133    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247135    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247138    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247140    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247143    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247146    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247148    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247151    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247153    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247156    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247159    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247161    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247163    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:28.251408 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247232    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247267    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247273    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247278    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247282    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247285    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247294    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247299    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247305    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247308    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247311    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247315    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247318    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247320    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247323    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247326    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247329    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247332    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247336    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247339    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247342    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247345    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247348    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247352    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 14:04:28.251915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247354    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247358    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247361    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247364    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247367    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247370    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247373    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247375    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247379    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247382    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247385    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247390    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247393    2580 flags.go:64] FLAG: --enable-server="true"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247396    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247400    2580 flags.go:64] FLAG: --event-burst="100"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247404    2580 flags.go:64] FLAG: --event-qps="50"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247406    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247410    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247413    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247417    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247420    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247423    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247425    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247428    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247431    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 14:04:28.252487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247434    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247437    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247440    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247443    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247446    2580 flags.go:64] FLAG: --feature-gates=""
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247451    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247454    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247457    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247460    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247463    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247466    2580 flags.go:64] FLAG: --help="false"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247468    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247472    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247474    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247477    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247480    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247483    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247486    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247490    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247493    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247496    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247499    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247502    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247505    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 16 14:04:28.253136 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247508    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247511    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247514    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247516    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247520    2580 flags.go:64] FLAG: --lock-file=""
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247522    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247525    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247528    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247533    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247536    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247538    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247541    2580 flags.go:64] FLAG: --logging-format="text"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247544    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247549    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247552    2580 flags.go:64] FLAG: --manifest-url=""
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247554    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247559    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247562    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247565    2580 flags.go:64] FLAG: --max-pods="110"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247568    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247571    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247574    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247577    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247580    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247583    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 14:04:28.253720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247586    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247594    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247597    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247600    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247603    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247605    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247610    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247613    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247616    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247619    2580 flags.go:64] FLAG: --port="10250"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247622    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247625    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dc946a6e48085e86"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247628    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247630    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247633    2580 flags.go:64] FLAG: --register-node="true"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247636    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247640    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247642    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247645    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247648    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247654    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247657    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247660    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247662    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247665    2580 flags.go:64] FLAG: --runonce="false"
Sep 16 14:04:28.254347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247668    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247671    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247674    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247677    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247680    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247683    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247686    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247689    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247692    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247696    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247698    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247701    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247704    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247707    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247710    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247715    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247718    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247726    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247732    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247735    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247738    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247740    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247743    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247746    2580 flags.go:64] FLAG: --v="2"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247750    2580 flags.go:64] FLAG: --version="false"
Sep 16 14:04:28.255043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247754    2580 flags.go:64] FLAG: --vmodule=""
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247758    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.247769    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247893    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247897    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247900    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247903    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247906    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247909    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247912    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247914    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247917    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247919    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247922    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247925    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247928    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247931    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247934    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247936    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247939    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:28.255669 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247942    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247944    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247947    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247950    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247952    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247960    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247964    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247967    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247970    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247973    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247976    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247978    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247981    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247983    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247987    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247990    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247992    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247995    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.247997    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:28.256168 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248000    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248002    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248005    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248007    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248009    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248012    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248015    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248017    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248020    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248024    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248026    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248029    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248032    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248036    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248040    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248043    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248047    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248050    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248053    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248057    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:28.256628 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248059    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248062    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248064    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248069    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248072    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248075    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248077    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248081    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248084    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248086    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248089    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248092    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248094    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248097    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248100    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248102    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248119    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248121    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248124    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:28.257169 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248127    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248130    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248134    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248136    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248140    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248143    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248146    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248149    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248152    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248155    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248158    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248160    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248163    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248169    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248172    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248175    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248177    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248179    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248182    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248184    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:28.257635 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248189    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248191    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248194    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.248196    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.248892    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.255978    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.255993    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.256614    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.256625    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256694    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256699    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256703    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256706    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256710    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:28.258142 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256713    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256716    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256719    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256722    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256725    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256727    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256730    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256732    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256735    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256737    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256740    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256742    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256745    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256747    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256750    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256752    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256754    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256757    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256760    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256763    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:28.258481 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256765    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256767    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256770    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256772    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256776    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256778    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256781    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256785    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256789    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256792    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256794    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256796    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256799    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256802    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256805    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256808    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256810    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256813    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256815    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256818    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:28.258966 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256820    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256823    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256826    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256829    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256831    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256834    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256836    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256839    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256841    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256844    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256846    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256849    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256851    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256853    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256856    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256858    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256861    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256864    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256867    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256870    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:28.259465 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256872    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256875    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256877    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256879    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256882    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256884    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256887    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256890    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256892    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256895    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256897    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256899    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256902    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256905    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256908    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256910    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256914    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256917    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256920    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:28.259934 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256924    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256927    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256930    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256932    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256935    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256937    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256940    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256942    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256944    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256947    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256949    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256952    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256954    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256957    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.256959    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:28.260439 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.256964    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257081    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257086    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257089    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257092    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257094    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257097    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257100    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257103    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257122    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257125    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257128    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257131    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257133    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257136    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257139    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257142    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257144    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257147    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:28.260795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257149    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257151    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257154    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257157    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257159    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257161    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257164    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257166    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257169    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257172    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257175    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257179    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257181    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257184    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257187    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257189    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257192    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257195    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257197    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257200    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:28.261263 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257202    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257205    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257208    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257211    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257215    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257218    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257220    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257223    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257226    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257229    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257231    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257234    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257236    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257239    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257241    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257244    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257246    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257248    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257251    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:28.261775 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257253    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257255    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257258    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257260    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257263    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257266    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257268    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257270    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257273    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257276    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257279    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257281    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257284    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257286    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257288    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257291    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257293    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257296    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257298    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257301    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:28.262247 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257303    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257306    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257308    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257311    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257313    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257315    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257318    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257320    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257323    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257325    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257327    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257330    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257332    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257335    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257339    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257344    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257346    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257349    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257352    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257354    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:28.262718 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257358    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:28.257361    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.257366    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.257479    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.259510    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.260435    2580 server.go:1019] "Starting client certificate rotation"
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.260531    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.260598    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:04:28.263227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.262827    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:04:28.264864 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.264813    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:04:28.277519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.277501    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 16 14:04:28.280848 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.280829    2580 log.go:25] "Validated CRI v1 image API"
Sep 16 14:04:28.281716 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.281700    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 14:04:28.286603 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.286583    2580 fs.go:135] Filesystem UUIDs: map[085974d4-cffc-4637-b338-ddc3679123d0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b57f68ec-50d0-49ba-abaf-59742bbd193a:/dev/nvme0n1p4]
Sep 16 14:04:28.286682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.286601    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 14:04:28.287663 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.287644    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:04:28.292233 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.292125    2580 manager.go:216] Machine: {Timestamp:2026-09-16 14:04:28.290968458 +0000 UTC m=+0.298008300 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203800 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2106c035ceccb729eff8996eced1c3 SystemUUID:ec2106c0-35ce-ccb7-29ef-f8996eced1c3 BootID:e23685f8-acf7-47be-a163-b7caf50a68b2 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e8:a1:2d:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e8:a1:2d:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:00:0f:bd:5b:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 14:04:28.292233 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.292226    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 14:04:28.292357 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.292302    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 14:04:28.294531 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.294505    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 14:04:28.294668 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.294532    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-34.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 14:04:28.295214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.295204    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 14:04:28.295245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.295216    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 14:04:28.295245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.295223    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:04:28.296366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.296355    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:04:28.297117 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.297098    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 14:04:28.301700 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.301689    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:04:28.301819 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.301809    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 14:04:28.301851 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.301839    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 14:04:28.301878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.301852    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 14:04:28.301878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.301867    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 14:04:28.301878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.301877    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 16 14:04:28.301967 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.301885    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 14:04:28.302264 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.302252    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:04:28.302309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.302270    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:04:28.304339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.304320    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 14:04:28.305964 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.305951    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 14:04:28.306003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.305980    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 14:04:28.307357 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307345    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 14:04:28.307397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307362    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 14:04:28.307397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307368    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 14:04:28.307397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307374    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 14:04:28.307397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307382    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 14:04:28.307397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307387    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 14:04:28.307536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307403    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 14:04:28.307536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307410    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 14:04:28.307536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307420    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 14:04:28.307536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307425    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 14:04:28.307536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307433    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 14:04:28.307536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.307438    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 14:04:28.308141 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.308131    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 14:04:28.308178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.308141    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 14:04:28.311899 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.311885    2580 server.go:1268] "Started kubelet"
Sep 16 14:04:28.312034 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.311988    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 14:04:28.312240 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.312156    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 14:04:28.312403 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.312366    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 14:04:28.312807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.312779    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-34.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 14:04:28.312960 ip-10-0-137-34 systemd[1]: Started Kubernetes Kubelet.
Sep 16 14:04:28.313330 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.313310    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 14:04:28.313330 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.313319    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 14:04:28.316992 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.315748    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 14:04:28.317831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.317350    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 14:04:28.319001 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.318970    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 14:04:28.319971 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.319950    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 14:04:28.320463 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.320452    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 14:04:28.320571 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.320545    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:28.321007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.320991    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 14:04:28.321007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.321008    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 14:04:28.321137 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.320992    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 14:04:28.321137 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.321086    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 14:04:28.321137 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.321092    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 14:04:28.321833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.321810    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xp6l5"
Sep 16 14:04:28.322030 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.321991    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 14:04:28.322588 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.322570    2580 factory.go:55] Registering systemd factory
Sep 16 14:04:28.322588 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.322589    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 16 14:04:28.323076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.323058    2580 factory.go:153] Registering CRI-O factory
Sep 16 14:04:28.323162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.323081    2580 factory.go:223] Registration of the crio container factory successfully
Sep 16 14:04:28.323162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.323150    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 14:04:28.323248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.323172    2580 factory.go:103] Registering Raw factory
Sep 16 14:04:28.323248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.323188    2580 manager.go:1196] Started watching for new ooms in manager
Sep 16 14:04:28.323569 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.323554    2580 manager.go:349] Starting recovery of all containers
Sep 16 14:04:28.325040 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.324241    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-34.ec2.internal.18d5d2218252b373  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-34.ec2.internal,UID:ip-10-0-137-34.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-34.ec2.internal,},FirstTimestamp:2026-09-16 14:04:28.311851891 +0000 UTC m=+0.318891737,LastTimestamp:2026-09-16 14:04:28.311851891 +0000 UTC m=+0.318891737,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-34.ec2.internal,}"
Sep 16 14:04:28.328814 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.328792    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xp6l5"
Sep 16 14:04:28.329549 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.329395    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 14:04:28.334426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.334409    2580 manager.go:354] Recovery completed
Sep 16 14:04:28.339971 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.339955    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:28.341744 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.341727    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:28.341812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.341757    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:28.341812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.341768    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:28.342421 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.342403    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 14:04:28.342421 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.342415    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:28.342533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.342418    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 14:04:28.342533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.342503    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:04:28.343087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.343072    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:28.343168 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.343096    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:28.343168 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.343127    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:28.344302 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.344130    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-34.ec2.internal.18d5d221841ad3c8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-34.ec2.internal,UID:ip-10-0-137-34.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-34.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-34.ec2.internal,},FirstTimestamp:2026-09-16 14:04:28.341744584 +0000 UTC m=+0.348784427,LastTimestamp:2026-09-16 14:04:28.341744584 +0000 UTC m=+0.348784427,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-34.ec2.internal,}"
Sep 16 14:04:28.344538 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.344521    2580 policy_none.go:49] "None policy: Start"
Sep 16 14:04:28.344577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.344548    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 14:04:28.344577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.344563    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 14:04:28.346607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.346595    2580 policy_none.go:47] "Start"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.378485    2580 manager.go:337] "Starting Device Plugin manager"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.378524    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.378536    2580 server.go:87] "Starting device plugin registration server"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.378877    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.378891    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.378994    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.379077    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.379087    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.380651    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 14:04:28.395568 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.380695    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:28.479007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.478979    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:28.479150 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.478978    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 14:04:28.480272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.480249    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 14:04:28.480339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.480310    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 14:04:28.480339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.480334    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 14:04:28.480482 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.480468    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 14:04:28.481051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.481032    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:28.481189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.481061    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:28.481189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.481075    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:28.481189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.481121    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.483307 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.483289    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:28.488391 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.488375    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.488483 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.488399    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-34.ec2.internal\": node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:28.553086 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.552991    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:28.581102 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.581072    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal"]
Sep 16 14:04:28.581194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.581184    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:28.584387 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.584370    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:28.584491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.584398    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:28.584491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.584408    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:28.586730 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.586716    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:28.586862 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.586837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.586919 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.586873    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:28.587558 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.587539    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:28.587657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.587568    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:28.587657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.587578    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:28.587657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.587539    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:28.587657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.587635    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:28.587657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.587646    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:28.589935 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.589920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.590004 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.589943    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:28.591345 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.591332    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:28.591404 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.591353    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:28.591404 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.591362    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:28.617728 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.617707    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-34.ec2.internal\" not found" node="ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.622172 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.622154    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-34.ec2.internal\" not found" node="ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.622254 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.622200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/13e1e09ae1d6a5cf42212bb85eac6f19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal\" (UID: \"13e1e09ae1d6a5cf42212bb85eac6f19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.622254 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.622229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13e1e09ae1d6a5cf42212bb85eac6f19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal\" (UID: \"13e1e09ae1d6a5cf42212bb85eac6f19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.622254 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.622244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bfb490af20cdfaeeb62e257a12524fb-config\") pod \"kube-apiserver-proxy-ip-10-0-137-34.ec2.internal\" (UID: \"2bfb490af20cdfaeeb62e257a12524fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.653238 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.653218    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:28.722558 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.722534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/13e1e09ae1d6a5cf42212bb85eac6f19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal\" (UID: \"13e1e09ae1d6a5cf42212bb85eac6f19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.722660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.722560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13e1e09ae1d6a5cf42212bb85eac6f19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal\" (UID: \"13e1e09ae1d6a5cf42212bb85eac6f19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.722660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.722577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bfb490af20cdfaeeb62e257a12524fb-config\") pod \"kube-apiserver-proxy-ip-10-0-137-34.ec2.internal\" (UID: \"2bfb490af20cdfaeeb62e257a12524fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.722660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.722639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bfb490af20cdfaeeb62e257a12524fb-config\") pod \"kube-apiserver-proxy-ip-10-0-137-34.ec2.internal\" (UID: \"2bfb490af20cdfaeeb62e257a12524fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.722660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.722638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13e1e09ae1d6a5cf42212bb85eac6f19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal\" (UID: \"13e1e09ae1d6a5cf42212bb85eac6f19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.722796 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.722638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/13e1e09ae1d6a5cf42212bb85eac6f19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal\" (UID: \"13e1e09ae1d6a5cf42212bb85eac6f19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.753674 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.753651    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:28.854479 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.854411    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:28.920630 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.920589    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.925649 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:28.925576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:28.955172 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:28.955144    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:29.055631 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:29.055591    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:30.675193 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:30.675099    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-34.ec2.internal\" not found"
Sep 16 14:04:30.682089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.682063    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:30.740671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.740630    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:30.754000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.753981    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:04:30.755559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.755546    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal"
Sep 16 14:04:30.765035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.765015    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:04:30.779190 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.779172    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 14:04:30.779295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.779276    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:04:30.779339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.779311    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:04:30.821484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.821461    2580 apiserver.go:52] "Watching apiserver"
Sep 16 14:04:30.836587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.836562    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 14:04:30.836906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.836887    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-txpr4","openshift-network-operator/iptables-alerter-p7frh","kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal","openshift-cluster-node-tuning-operator/tuned-zgdjq","openshift-dns/node-resolver-pv8ks","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal","openshift-multus/multus-additional-cni-plugins-fx94s","openshift-multus/network-metrics-daemon-pq29s","openshift-network-diagnostics/network-check-target-d96cv","openshift-ovn-kubernetes/ovnkube-node-vwwx4","kube-system/global-pull-secret-syncer-px6lq","kube-system/konnectivity-agent-5jpt6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7","openshift-image-registry/node-ca-mns9d"]
Sep 16 14:04:30.839346 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.839331    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 14:04:30.839467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.839454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.842233 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.842208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.843209 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.843195    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 14:04:30.843288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.843259    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xj5ng\""
Sep 16 14:04:30.843354 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.843320    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 14:04:30.843478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.843463    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 14:04:30.843549 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.843507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 14:04:30.844237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.844358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-cni-bin\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-daemon-config\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf58899c-5f52-4b1c-82a9-2adc1b62999d-iptables-alerter-script\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.844426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf58899c-5f52-4b1c-82a9-2adc1b62999d-host-slash\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.844544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-kubelet\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-system-cni-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-cni-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-cnibin\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq4kz\" (UniqueName: \"kubernetes.io/projected/cf58899c-5f52-4b1c-82a9-2adc1b62999d-kube-api-access-lq4kz\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.844544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-socket-dir-parent\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-netns\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-hostroot\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-conf-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-multus-certs\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-cni-binary-copy\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-k8s-cni-cncf-io\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-cni-multus\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-etc-kubernetes\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.844771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.844734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm2sw\" (UniqueName: \"kubernetes.io/projected/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-kube-api-access-sm2sw\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.847737 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.847715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:30.848503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.848483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 14:04:30.848586 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.848545    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:04:30.848656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.848606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:04:30.848709 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.848654    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d6ttt\""
Sep 16 14:04:30.848765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.848656    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 14:04:30.849499 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.849480    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tpfr6\""
Sep 16 14:04:30.849583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.849526    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 14:04:30.850095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.850074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.852216 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.852192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:30.852309 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:30.852259    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:30.852483 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.852444    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 13:59:29 +0000 UTC" deadline="2028-05-23 09:07:46.321487763 +0000 UTC"
Sep 16 14:04:30.852549 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.852513    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14755h3m15.469009047s"
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.854451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:30.854510    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.854548    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-swwlx\""
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.854579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.854630    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.854644    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.854644    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mgvg9\""
Sep 16 14:04:30.854736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.854548    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 14:04:30.856700 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.856683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.859068 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.859045    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 14:04:30.859177 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.859071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 14:04:30.859177 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.859096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:30.859344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.859046    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 14:04:30.859344 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:30.859182    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:30.859344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.859290    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 14:04:30.859344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.859296    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9cqmq\""
Sep 16 14:04:30.860401 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.860383    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 14:04:30.860481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.860444    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 14:04:30.861382 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.861366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:30.862643 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.862628    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:04:30.863721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.863701    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 14:04:30.863721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.863710    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 14:04:30.863862 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.863722    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hfsls\""
Sep 16 14:04:30.864284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.864267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.867948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.866918    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 14:04:30.867948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.866981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-grknj\""
Sep 16 14:04:30.867948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.867544    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 14:04:30.867948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.867577    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 14:04:30.868583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.868567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:30.870735 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.870715    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 14:04:30.871361 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.871345    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 14:04:30.871462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.871367    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 14:04:30.871462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.871379    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4fxz6\""
Sep 16 14:04:30.888358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.888342    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-blskj"
Sep 16 14:04:30.895827 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.895808    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-blskj"
Sep 16 14:04:30.941028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.941010    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 14:04:30.945042 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf58899c-5f52-4b1c-82a9-2adc1b62999d-iptables-alerter-script\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.945165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysconfig\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.945165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-host\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.945165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72c7f4b-fbd4-496b-b66c-800629e3a75e-tmp-dir\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:30.945165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e8f9027-f385-4e8d-a1dd-f4829d480d77-konnectivity-ca\") pod \"konnectivity-agent-5jpt6\" (UID: \"0e8f9027-f385-4e8d-a1dd-f4829d480d77\") " pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-kubelet\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-cni-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-serviceca\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-kubernetes\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-tuned\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-kubelet\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-cni-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.945326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-tmp\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-slash\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzff4\" (UniqueName: \"kubernetes.io/projected/206c7143-17a7-46b9-93b9-c1509d26ccc1-kube-api-access-gzff4\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-etc-kubernetes\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-systemd\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945513    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-etc-kubernetes\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-run\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf58899c-5f52-4b1c-82a9-2adc1b62999d-iptables-alerter-script\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjqhf\" (UniqueName: \"kubernetes.io/projected/24c0e5cb-4a9e-4377-8e92-61022a699d47-kube-api-access-kjqhf\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-var-lib-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-etc-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.945662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-cni-netd\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovnkube-script-lib\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pndhk\" (UniqueName: \"kubernetes.io/projected/c8c153ad-02b2-4d61-956b-2bded0db29a6-kube-api-access-pndhk\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-cnibin\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-systemd\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-registration-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-kubelet-config\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysctl-d\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovn-node-metrics-cert\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-etc-selinux\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-sys-fs\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.945980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-k8s-cni-cncf-io\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-device-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-k8s-cni-cncf-io\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e8f9027-f385-4e8d-a1dd-f4829d480d77-agent-certs\") pod \"konnectivity-agent-5jpt6\" (UID: \"0e8f9027-f385-4e8d-a1dd-f4829d480d77\") " pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m6pv\" (UniqueName: \"kubernetes.io/projected/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-kube-api-access-9m6pv\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:30.946162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-daemon-config\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf58899c-5f52-4b1c-82a9-2adc1b62999d-host-slash\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf58899c-5f52-4b1c-82a9-2adc1b62999d-host-slash\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-modprobe-d\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-log-socket\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-cnibin\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-kubelet\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-socket-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-cnibin\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-dbus\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-netns\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-conf-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-netns\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.946883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsqdq\" (UniqueName: \"kubernetes.io/projected/d72c7f4b-fbd4-496b-b66c-800629e3a75e-kube-api-access-qsqdq\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-run-netns\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-conf-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-node-log\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-cni-binary-copy\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-cni-multus\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm2sw\" (UniqueName: \"kubernetes.io/projected/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-kube-api-access-sm2sw\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-var-lib-kubelet\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-daemon-config\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-cni-multus\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovnkube-config\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-system-cni-dir\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-cni-bin\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysctl-conf\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946863    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-var-lib-cni-bin\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-sys\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.947598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d72c7f4b-fbd4-496b-b66c-800629e3a75e-hosts-file\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-ovn\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zlhl\" (UniqueName: \"kubernetes.io/projected/82cd6309-31f0-45bf-913d-869248730c15-kube-api-access-5zlhl\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.946999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-cni-binary-copy\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-system-cni-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq4kz\" (UniqueName: \"kubernetes.io/projected/cf58899c-5f52-4b1c-82a9-2adc1b62999d-kube-api-access-lq4kz\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-lib-modules\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-systemd-units\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-system-cni-dir\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-env-overrides\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-socket-dir-parent\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-hostroot\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-multus-certs\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-hostroot\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-multus-socket-dir-parent\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9qph\" (UniqueName: \"kubernetes.io/projected/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-kube-api-access-c9qph\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:30.948530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-host-run-multus-certs\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.948530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:30.948530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:30.948530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-host\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:30.948530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-cni-bin\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:30.948530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.947451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:30.952783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.952764    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 14:04:30.955905 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.955880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm2sw\" (UniqueName: \"kubernetes.io/projected/984c2616-a7ee-4b42-ae38-bac0e2aa1e5a-kube-api-access-sm2sw\") pod \"multus-txpr4\" (UID: \"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a\") " pod="openshift-multus/multus-txpr4"
Sep 16 14:04:30.955988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:30.955906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq4kz\" (UniqueName: \"kubernetes.io/projected/cf58899c-5f52-4b1c-82a9-2adc1b62999d-kube-api-access-lq4kz\") pod \"iptables-alerter-p7frh\" (UID: \"cf58899c-5f52-4b1c-82a9-2adc1b62999d\") " pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:31.048038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-host\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:31.048038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-cni-bin\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysconfig\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-host\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-cni-bin\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-host\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72c7f4b-fbd4-496b-b66c-800629e3a75e-tmp-dir\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048195    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysconfig\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048301 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.048206    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-host\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e8f9027-f385-4e8d-a1dd-f4829d480d77-konnectivity-ca\") pod \"konnectivity-agent-5jpt6\" (UID: \"0e8f9027-f385-4e8d-a1dd-f4829d480d77\") " pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.048356    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs podName:24c0e5cb-4a9e-4377-8e92-61022a699d47 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:31.548323734 +0000 UTC m=+2.036333804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs") pod "network-metrics-daemon-pq29s" (UID: "24c0e5cb-4a9e-4377-8e92-61022a699d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72c7f4b-fbd4-496b-b66c-800629e3a75e-tmp-dir\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-serviceca\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-kubernetes\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-tuned\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-tmp\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-slash\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-kubernetes\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-slash\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.048685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e8f9027-f385-4e8d-a1dd-f4829d480d77-konnectivity-ca\") pod \"konnectivity-agent-5jpt6\" (UID: \"0e8f9027-f385-4e8d-a1dd-f4829d480d77\") " pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzff4\" (UniqueName: \"kubernetes.io/projected/206c7143-17a7-46b9-93b9-c1509d26ccc1-kube-api-access-gzff4\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-systemd\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-systemd\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-run\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjqhf\" (UniqueName: \"kubernetes.io/projected/24c0e5cb-4a9e-4377-8e92-61022a699d47-kube-api-access-kjqhf\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048984    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-serviceca\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-run\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.048998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-var-lib-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-var-lib-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-etc-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-etc-openvswitch\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-cni-netd\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovnkube-script-lib\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-cni-netd\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.049237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pndhk\" (UniqueName: \"kubernetes.io/projected/c8c153ad-02b2-4d61-956b-2bded0db29a6-kube-api-access-pndhk\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-cnibin\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-systemd\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-registration-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-cnibin\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-kubelet-config\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-systemd\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-registration-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysctl-d\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-kubelet-config\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovn-node-metrics-cert\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-etc-selinux\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-sys-fs\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-device-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e8f9027-f385-4e8d-a1dd-f4829d480d77-agent-certs\") pod \"konnectivity-agent-5jpt6\" (UID: \"0e8f9027-f385-4e8d-a1dd-f4829d480d77\") " pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049531    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysctl-d\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-sys-fs\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6pv\" (UniqueName: \"kubernetes.io/projected/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-kube-api-access-9m6pv\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-device-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-etc-selinux\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-modprobe-d\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-log-socket\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovnkube-script-lib\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-kubelet\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-socket-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-dbus\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsqdq\" (UniqueName: \"kubernetes.io/projected/d72c7f4b-fbd4-496b-b66c-800629e3a75e-kube-api-access-qsqdq\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.049996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-run-netns\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-node-log\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-var-lib-kubelet\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.050958 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.050089    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovnkube-config\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-system-cni-dir\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysctl-conf\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.050249    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret podName:dd12c3f1-620d-4992-bb8b-68c1293ab5ab nodeName:}" failed. No retries permitted until 2026-09-16 14:04:31.55022235 +0000 UTC m=+2.038232419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret") pod "global-pull-secret-syncer-px6lq" (UID: "dd12c3f1-620d-4992-bb8b-68c1293ab5ab") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-sys\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d72c7f4b-fbd4-496b-b66c-800629e3a75e-hosts-file\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-ovn\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-sysctl-conf\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-node-log\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zlhl\" (UniqueName: \"kubernetes.io/projected/82cd6309-31f0-45bf-913d-869248730c15-kube-api-access-5zlhl\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-var-lib-kubelet\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-lib-modules\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.051767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-log-socket\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-systemd-units\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-env-overrides\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9qph\" (UniqueName: \"kubernetes.io/projected/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-kube-api-access-c9qph\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-socket-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-modprobe-d\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-system-cni-dir\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-run-ovn\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-run-netns\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-lib-modules\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-systemd-units\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-etc-tuned\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.052473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.050546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8c153ad-02b2-4d61-956b-2bded0db29a6-host-kubelet\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovnkube-config\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c8c153ad-02b2-4d61-956b-2bded0db29a6-env-overrides\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-dbus\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051506    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/82cd6309-31f0-45bf-913d-869248730c15-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-sys\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d72c7f4b-fbd4-496b-b66c-800629e3a75e-hosts-file\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/206c7143-17a7-46b9-93b9-c1509d26ccc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8c153ad-02b2-4d61-956b-2bded0db29a6-ovn-node-metrics-cert\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.051819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/206c7143-17a7-46b9-93b9-c1509d26ccc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.052108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-tmp\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.052999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.052229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e8f9027-f385-4e8d-a1dd-f4829d480d77-agent-certs\") pod \"konnectivity-agent-5jpt6\" (UID: \"0e8f9027-f385-4e8d-a1dd-f4829d480d77\") " pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:31.056367 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.056348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzff4\" (UniqueName: \"kubernetes.io/projected/206c7143-17a7-46b9-93b9-c1509d26ccc1-kube-api-access-gzff4\") pod \"multus-additional-cni-plugins-fx94s\" (UID: \"206c7143-17a7-46b9-93b9-c1509d26ccc1\") " pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.057567 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.057546    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:31.057567 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.057568    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:31.057724 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.057578    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cvlrg for pod openshift-network-diagnostics/network-check-target-d96cv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:31.057724 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.057647    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg podName:603ccdba-2511-4336-9480-1fa9490508d8 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:31.557633907 +0000 UTC m=+2.045643975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cvlrg" (UniqueName: "kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg") pod "network-check-target-d96cv" (UID: "603ccdba-2511-4336-9480-1fa9490508d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:31.062560 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.060403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsqdq\" (UniqueName: \"kubernetes.io/projected/d72c7f4b-fbd4-496b-b66c-800629e3a75e-kube-api-access-qsqdq\") pod \"node-resolver-pv8ks\" (UID: \"d72c7f4b-fbd4-496b-b66c-800629e3a75e\") " pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:31.062560 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.060633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjqhf\" (UniqueName: \"kubernetes.io/projected/24c0e5cb-4a9e-4377-8e92-61022a699d47-kube-api-access-kjqhf\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:31.062560 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.060762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zlhl\" (UniqueName: \"kubernetes.io/projected/82cd6309-31f0-45bf-913d-869248730c15-kube-api-access-5zlhl\") pod \"aws-ebs-csi-driver-node-thkw7\" (UID: \"82cd6309-31f0-45bf-913d-869248730c15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.063208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.063188    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9qph\" (UniqueName: \"kubernetes.io/projected/9758160b-dbeb-4a30-9f3e-f2e3f491aea5-kube-api-access-c9qph\") pod \"tuned-zgdjq\" (UID: \"9758160b-dbeb-4a30-9f3e-f2e3f491aea5\") " pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.064035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.063994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pndhk\" (UniqueName: \"kubernetes.io/projected/c8c153ad-02b2-4d61-956b-2bded0db29a6-kube-api-access-pndhk\") pod \"ovnkube-node-vwwx4\" (UID: \"c8c153ad-02b2-4d61-956b-2bded0db29a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.064035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.064029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m6pv\" (UniqueName: \"kubernetes.io/projected/a01415cc-8ef7-4e60-8deb-0b314dd4fe3d-kube-api-access-9m6pv\") pod \"node-ca-mns9d\" (UID: \"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d\") " pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:31.079552 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.079524    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e1e09ae1d6a5cf42212bb85eac6f19.slice/crio-ae18e318a842aca82792eac2fb786af62fa60bdc24b65742f119dbb2318d1005 WatchSource:0}: Error finding container ae18e318a842aca82792eac2fb786af62fa60bdc24b65742f119dbb2318d1005: Status 404 returned error can't find the container with id ae18e318a842aca82792eac2fb786af62fa60bdc24b65742f119dbb2318d1005
Sep 16 14:04:31.079844 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.079825    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfb490af20cdfaeeb62e257a12524fb.slice/crio-d17e0c016e9f6d69cd23ddef15e2a43f19d3828f48d7bfa0629c530c0c5d5de9 WatchSource:0}: Error finding container d17e0c016e9f6d69cd23ddef15e2a43f19d3828f48d7bfa0629c530c0c5d5de9: Status 404 returned error can't find the container with id d17e0c016e9f6d69cd23ddef15e2a43f19d3828f48d7bfa0629c530c0c5d5de9
Sep 16 14:04:31.083460 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.083442    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:04:31.109498 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.109474    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:31.159084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.159044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-txpr4"
Sep 16 14:04:31.168282 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.168257    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984c2616_a7ee_4b42_ae38_bac0e2aa1e5a.slice/crio-0bd8383306fa0f01d70893349c4a719b3ac1fcbc668ecfdb16d982a57a55a135 WatchSource:0}: Error finding container 0bd8383306fa0f01d70893349c4a719b3ac1fcbc668ecfdb16d982a57a55a135: Status 404 returned error can't find the container with id 0bd8383306fa0f01d70893349c4a719b3ac1fcbc668ecfdb16d982a57a55a135
Sep 16 14:04:31.175347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.175329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p7frh"
Sep 16 14:04:31.183566 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.183513    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf58899c_5f52_4b1c_82a9_2adc1b62999d.slice/crio-ee073233032c8e60f4b3e86afee755af508c4dc9e23c9c175846cbdff171dc1b WatchSource:0}: Error finding container ee073233032c8e60f4b3e86afee755af508c4dc9e23c9c175846cbdff171dc1b: Status 404 returned error can't find the container with id ee073233032c8e60f4b3e86afee755af508c4dc9e23c9c175846cbdff171dc1b
Sep 16 14:04:31.199376 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.199356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zgdjq"
Sep 16 14:04:31.206752 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.206732    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9758160b_dbeb_4a30_9f3e_f2e3f491aea5.slice/crio-128134b27abf8a3de2d3905fa4d42787c1fbf98d99dc6b8aa9994efa0167906b WatchSource:0}: Error finding container 128134b27abf8a3de2d3905fa4d42787c1fbf98d99dc6b8aa9994efa0167906b: Status 404 returned error can't find the container with id 128134b27abf8a3de2d3905fa4d42787c1fbf98d99dc6b8aa9994efa0167906b
Sep 16 14:04:31.208573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.208557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pv8ks"
Sep 16 14:04:31.213744 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.213721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fx94s"
Sep 16 14:04:31.216209 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.216181    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd72c7f4b_fbd4_496b_b66c_800629e3a75e.slice/crio-7a9e30e9b40fd32ed142ddda94bc86d4b3f74910e6fe2cb7ffd87aee6ec95c04 WatchSource:0}: Error finding container 7a9e30e9b40fd32ed142ddda94bc86d4b3f74910e6fe2cb7ffd87aee6ec95c04: Status 404 returned error can't find the container with id 7a9e30e9b40fd32ed142ddda94bc86d4b3f74910e6fe2cb7ffd87aee6ec95c04
Sep 16 14:04:31.219922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.219903    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:31.222178 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.222156    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206c7143_17a7_46b9_93b9_c1509d26ccc1.slice/crio-b5877d20aac1badf8c00b17915d1908d4c94633332b23dfa7b91b1f5aa7f1e24 WatchSource:0}: Error finding container b5877d20aac1badf8c00b17915d1908d4c94633332b23dfa7b91b1f5aa7f1e24: Status 404 returned error can't find the container with id b5877d20aac1badf8c00b17915d1908d4c94633332b23dfa7b91b1f5aa7f1e24
Sep 16 14:04:31.228773 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.228755    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c153ad_02b2_4d61_956b_2bded0db29a6.slice/crio-738a7d9aa76ae7d403949664b73fd85e4c6bc83c2b4889802bd13ca16a845afb WatchSource:0}: Error finding container 738a7d9aa76ae7d403949664b73fd85e4c6bc83c2b4889802bd13ca16a845afb: Status 404 returned error can't find the container with id 738a7d9aa76ae7d403949664b73fd85e4c6bc83c2b4889802bd13ca16a845afb
Sep 16 14:04:31.239736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.239719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:31.247683 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.247664    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e8f9027_f385_4e8d_a1dd_f4829d480d77.slice/crio-7fd12df686f1aea4ef6ac67f9c55530565ec81a8ecee79e398f96f4b25941ac5 WatchSource:0}: Error finding container 7fd12df686f1aea4ef6ac67f9c55530565ec81a8ecee79e398f96f4b25941ac5: Status 404 returned error can't find the container with id 7fd12df686f1aea4ef6ac67f9c55530565ec81a8ecee79e398f96f4b25941ac5
Sep 16 14:04:31.265103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.265078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7"
Sep 16 14:04:31.270878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.270864    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mns9d"
Sep 16 14:04:31.272503 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.272484    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82cd6309_31f0_45bf_913d_869248730c15.slice/crio-b9b82a5472d98a092087edd532353ffb9f535fefd5e69d4e2e1563a02eccb3f9 WatchSource:0}: Error finding container b9b82a5472d98a092087edd532353ffb9f535fefd5e69d4e2e1563a02eccb3f9: Status 404 returned error can't find the container with id b9b82a5472d98a092087edd532353ffb9f535fefd5e69d4e2e1563a02eccb3f9
Sep 16 14:04:31.279344 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:04:31.279326    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01415cc_8ef7_4e60_8deb_0b314dd4fe3d.slice/crio-8443ee6444c320d9bc0b8d7da581dbcdd7b605f66af2512b11cce66034798cf0 WatchSource:0}: Error finding container 8443ee6444c320d9bc0b8d7da581dbcdd7b605f66af2512b11cce66034798cf0: Status 404 returned error can't find the container with id 8443ee6444c320d9bc0b8d7da581dbcdd7b605f66af2512b11cce66034798cf0
Sep 16 14:04:31.280645 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.280631    2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container node-ca start failed in pod node-ca-mns9d_openshift-image-registry(a01415cc-8ef7-4e60-8deb-0b314dd4fe3d): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 16 14:04:31.280704 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.280657    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node-ca\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-image-registry/node-ca-mns9d" podUID="a01415cc-8ef7-4e60-8deb-0b314dd4fe3d"
Sep 16 14:04:31.289523 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.289506    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:31.553242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.553159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:31.553242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.553223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:31.553428 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.553334    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:31.553428 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.553349    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:31.553428 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.553403    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret podName:dd12c3f1-620d-4992-bb8b-68c1293ab5ab nodeName:}" failed. No retries permitted until 2026-09-16 14:04:32.553387419 +0000 UTC m=+3.041397477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret") pod "global-pull-secret-syncer-px6lq" (UID: "dd12c3f1-620d-4992-bb8b-68c1293ab5ab") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:31.553428 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.553418    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs podName:24c0e5cb-4a9e-4377-8e92-61022a699d47 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:32.553412048 +0000 UTC m=+3.041422102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs") pod "network-metrics-daemon-pq29s" (UID: "24c0e5cb-4a9e-4377-8e92-61022a699d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:31.654825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.654214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:31.654825 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.654411    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:31.654825 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.654432    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:31.654825 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.654444    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cvlrg for pod openshift-network-diagnostics/network-check-target-d96cv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:31.654825 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:31.654503    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg podName:603ccdba-2511-4336-9480-1fa9490508d8 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:32.654486385 +0000 UTC m=+3.142496444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvlrg" (UniqueName: "kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg") pod "network-check-target-d96cv" (UID: "603ccdba-2511-4336-9480-1fa9490508d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:31.871842 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.871576    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:31.898847 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.898734    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:59:30 +0000 UTC" deadline="2028-05-14 03:24:20.946801583 +0000 UTC"
Sep 16 14:04:31.898847 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:31.898789    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14533h19m49.048016973s"
Sep 16 14:04:32.021109 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.021053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-txpr4" event={"ID":"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a","Type":"ContainerStarted","Data":"0bd8383306fa0f01d70893349c4a719b3ac1fcbc668ecfdb16d982a57a55a135"}
Sep 16 14:04:32.023101 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.023071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal" event={"ID":"2bfb490af20cdfaeeb62e257a12524fb","Type":"ContainerStarted","Data":"d17e0c016e9f6d69cd23ddef15e2a43f19d3828f48d7bfa0629c530c0c5d5de9"}
Sep 16 14:04:32.028281 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.028238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal" event={"ID":"13e1e09ae1d6a5cf42212bb85eac6f19","Type":"ContainerStarted","Data":"ae18e318a842aca82792eac2fb786af62fa60bdc24b65742f119dbb2318d1005"}
Sep 16 14:04:32.037029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.037000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5jpt6" event={"ID":"0e8f9027-f385-4e8d-a1dd-f4829d480d77","Type":"ContainerStarted","Data":"7fd12df686f1aea4ef6ac67f9c55530565ec81a8ecee79e398f96f4b25941ac5"}
Sep 16 14:04:32.040991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.040965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"738a7d9aa76ae7d403949664b73fd85e4c6bc83c2b4889802bd13ca16a845afb"}
Sep 16 14:04:32.053196 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.053171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerStarted","Data":"b5877d20aac1badf8c00b17915d1908d4c94633332b23dfa7b91b1f5aa7f1e24"}
Sep 16 14:04:32.061060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.061036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pv8ks" event={"ID":"d72c7f4b-fbd4-496b-b66c-800629e3a75e","Type":"ContainerStarted","Data":"7a9e30e9b40fd32ed142ddda94bc86d4b3f74910e6fe2cb7ffd87aee6ec95c04"}
Sep 16 14:04:32.069641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.069612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zgdjq" event={"ID":"9758160b-dbeb-4a30-9f3e-f2e3f491aea5","Type":"ContainerStarted","Data":"128134b27abf8a3de2d3905fa4d42787c1fbf98d99dc6b8aa9994efa0167906b"}
Sep 16 14:04:32.081887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.081830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p7frh" event={"ID":"cf58899c-5f52-4b1c-82a9-2adc1b62999d","Type":"ContainerStarted","Data":"ee073233032c8e60f4b3e86afee755af508c4dc9e23c9c175846cbdff171dc1b"}
Sep 16 14:04:32.084841 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.084813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mns9d" event={"ID":"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d","Type":"ContainerStarted","Data":"8443ee6444c320d9bc0b8d7da581dbcdd7b605f66af2512b11cce66034798cf0"}
Sep 16 14:04:32.089879 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.089545    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node-ca\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1ede51f0939b0c3d64025a444de8cbd252789c4c2a600a6301081f72ac78aeb7\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-image-registry/node-ca-mns9d" podUID="a01415cc-8ef7-4e60-8deb-0b314dd4fe3d"
Sep 16 14:04:32.093283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.093216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7" event={"ID":"82cd6309-31f0-45bf-913d-869248730c15","Type":"ContainerStarted","Data":"b9b82a5472d98a092087edd532353ffb9f535fefd5e69d4e2e1563a02eccb3f9"}
Sep 16 14:04:32.562259 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.562211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:32.562442 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.562315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:32.562442 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.562387    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:32.562442 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.562436    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:32.562597 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.562460    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs podName:24c0e5cb-4a9e-4377-8e92-61022a699d47 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:34.562441491 +0000 UTC m=+5.050451558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs") pod "network-metrics-daemon-pq29s" (UID: "24c0e5cb-4a9e-4377-8e92-61022a699d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:32.562597 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.562485    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret podName:dd12c3f1-620d-4992-bb8b-68c1293ab5ab nodeName:}" failed. No retries permitted until 2026-09-16 14:04:34.562469798 +0000 UTC m=+5.050479869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret") pod "global-pull-secret-syncer-px6lq" (UID: "dd12c3f1-620d-4992-bb8b-68c1293ab5ab") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:32.662924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.662891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:32.663106 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.663094    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:32.663190 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.663115    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:32.663190 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.663126    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cvlrg for pod openshift-network-diagnostics/network-check-target-d96cv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:32.663302 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:32.663199    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg podName:603ccdba-2511-4336-9480-1fa9490508d8 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:34.663178763 +0000 UTC m=+5.151188837 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvlrg" (UniqueName: "kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg") pod "network-check-target-d96cv" (UID: "603ccdba-2511-4336-9480-1fa9490508d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:32.899808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.899715    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:59:30 +0000 UTC" deadline="2028-02-12 00:01:11.258331283 +0000 UTC"
Sep 16 14:04:32.899808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:32.899760    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12321h56m38.358575984s"
Sep 16 14:04:33.000086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:33.000053    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:33.000284 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:33.000214    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:33.000729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:33.000607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:33.000855 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:33.000789    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:33.000855 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:33.000821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:33.000980 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:33.000891    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:33.087647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:33.087504    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:34.580076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:34.579047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:34.580076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:34.579388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:34.580076 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.579266    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:34.580076 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.579615    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret podName:dd12c3f1-620d-4992-bb8b-68c1293ab5ab nodeName:}" failed. No retries permitted until 2026-09-16 14:04:38.579569486 +0000 UTC m=+9.067579560 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret") pod "global-pull-secret-syncer-px6lq" (UID: "dd12c3f1-620d-4992-bb8b-68c1293ab5ab") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:34.580076 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.579524    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:34.580076 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.580048    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs podName:24c0e5cb-4a9e-4377-8e92-61022a699d47 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:38.580029764 +0000 UTC m=+9.068039839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs") pod "network-metrics-daemon-pq29s" (UID: "24c0e5cb-4a9e-4377-8e92-61022a699d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:34.680117 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:34.679950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:34.680312 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.680184    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:34.680312 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.680209    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:34.680312 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.680220    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cvlrg for pod openshift-network-diagnostics/network-check-target-d96cv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:34.680312 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.680280    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg podName:603ccdba-2511-4336-9480-1fa9490508d8 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:38.680262516 +0000 UTC m=+9.168272586 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvlrg" (UniqueName: "kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg") pod "network-check-target-d96cv" (UID: "603ccdba-2511-4336-9480-1fa9490508d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:34.999889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:34.999855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:35.000070 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:34.999983    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:35.000473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:35.000208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:35.000473 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:35.000304    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:35.000473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:35.000363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:35.000473 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:35.000442    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:37.000729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:37.000461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:37.000729 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:37.000580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:37.001274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:37.000780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:37.001274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:37.000781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:37.001274 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:37.000898    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:37.001274 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:37.000982    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:38.616826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:38.611476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:38.616826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:38.611533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:38.616826 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.613370    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:38.616826 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.613440    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret podName:dd12c3f1-620d-4992-bb8b-68c1293ab5ab nodeName:}" failed. No retries permitted until 2026-09-16 14:04:46.61342524 +0000 UTC m=+17.101435294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret") pod "global-pull-secret-syncer-px6lq" (UID: "dd12c3f1-620d-4992-bb8b-68c1293ab5ab") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:38.616826 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.613912    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:38.616826 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.613962    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs podName:24c0e5cb-4a9e-4377-8e92-61022a699d47 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:46.613944457 +0000 UTC m=+17.101954511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs") pod "network-metrics-daemon-pq29s" (UID: "24c0e5cb-4a9e-4377-8e92-61022a699d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:38.712732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:38.712659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:38.713018 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.712988    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:38.713018 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.713014    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:38.713190 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.713028    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cvlrg for pod openshift-network-diagnostics/network-check-target-d96cv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:38.713190 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:38.713099    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg podName:603ccdba-2511-4336-9480-1fa9490508d8 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:46.71307508 +0000 UTC m=+17.201085138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvlrg" (UniqueName: "kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg") pod "network-check-target-d96cv" (UID: "603ccdba-2511-4336-9480-1fa9490508d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:39.001290 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:39.000386    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:39.001290 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:39.000528    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:39.001290 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:39.000934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:39.001290 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:39.001068    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:39.001290 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:39.001152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:39.001290 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:39.001237    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:40.999723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:40.999687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:41.000166 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:40.999801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:41.000166 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:40.999813    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:41.000166 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:40.999830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:41.000166 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:40.999908    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:41.000166 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:40.999980    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:43.000626 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:43.000570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:43.000626 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:43.000592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:43.001215 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:43.000779    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:43.001215 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:43.000951    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:43.001215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:43.001021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:43.001898 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:43.001854    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:44.999599 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:44.999564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:45.000080 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:44.999608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:45.000080 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:44.999675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:45.000080 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:44.999690    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:45.000080 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:44.999795    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:45.000080 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:44.999881    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:46.669356 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:46.669325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:46.669836 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:46.669366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:46.669836 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.669479    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:46.669836 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.669497    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:46.669836 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.669543    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret podName:dd12c3f1-620d-4992-bb8b-68c1293ab5ab nodeName:}" failed. No retries permitted until 2026-09-16 14:05:02.669524086 +0000 UTC m=+33.157534156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret") pod "global-pull-secret-syncer-px6lq" (UID: "dd12c3f1-620d-4992-bb8b-68c1293ab5ab") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:46.669836 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.669562    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs podName:24c0e5cb-4a9e-4377-8e92-61022a699d47 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:02.669553369 +0000 UTC m=+33.157563423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs") pod "network-metrics-daemon-pq29s" (UID: "24c0e5cb-4a9e-4377-8e92-61022a699d47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:46.769982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:46.769943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:46.770160 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.770115    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:46.770160 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.770152    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:46.770279 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.770165    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cvlrg for pod openshift-network-diagnostics/network-check-target-d96cv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:46.770279 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:46.770226    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg podName:603ccdba-2511-4336-9480-1fa9490508d8 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:02.770208911 +0000 UTC m=+33.258218979 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvlrg" (UniqueName: "kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg") pod "network-check-target-d96cv" (UID: "603ccdba-2511-4336-9480-1fa9490508d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:46.999939 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:46.999903    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:46.999939 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:46.999927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:47.000254 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:46.999927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:47.000254 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:47.000043    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:47.000254 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:47.000198    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:47.000405 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:47.000278    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:49.000451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.000409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:49.000451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.000430    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:49.001336 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:49.000559    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:49.001336 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:49.000634    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:49.001336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.000696    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:49.001336 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:49.000768    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:49.136783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.136488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7" event={"ID":"82cd6309-31f0-45bf-913d-869248730c15","Type":"ContainerStarted","Data":"d6566cf21d0f6a42f3f996b1da6eb22bbfbe700f9ab4deee3eb4208ac34ea94a"}
Sep 16 14:04:49.139390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.139329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-txpr4" event={"ID":"984c2616-a7ee-4b42-ae38-bac0e2aa1e5a","Type":"ContainerStarted","Data":"756f08e1b67094491454f51e6eeafb837cb100bc136825381388ed96ac5a122d"}
Sep 16 14:04:49.147786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.143777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal" event={"ID":"2bfb490af20cdfaeeb62e257a12524fb","Type":"ContainerStarted","Data":"3335764c627297e73a01af8b269648b069739580889ec2ecece73a22b4978fcd"}
Sep 16 14:04:49.147786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.145589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal" event={"ID":"13e1e09ae1d6a5cf42212bb85eac6f19","Type":"ContainerStarted","Data":"0cf9a896e08384490a95dd568c6a4a6b638f1a80e4042035346f3cdcab9e62eb"}
Sep 16 14:04:49.150297 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.149664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5jpt6" event={"ID":"0e8f9027-f385-4e8d-a1dd-f4829d480d77","Type":"ContainerStarted","Data":"0e59a873b972e72c927845f43608c34c958c9d2c47b2f7727581adc796f87da9"}
Sep 16 14:04:49.152692 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.152565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"18f48d1a9d176b7a2d57fd7d08e658741e6f8e4c63cdc112209af3c30eaf20d1"}
Sep 16 14:04:49.152692 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.152637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"1b8eba88986974e1526d2b5f0f076ee44ac88f78babe4c5ee8e639e3b9d6d7a5"}
Sep 16 14:04:49.154722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.154535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerStarted","Data":"c2684cee1b5b68658c458fd8800d45f20f4c5deba4dde5aa124344f0b848d580"}
Sep 16 14:04:49.158314 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.158215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pv8ks" event={"ID":"d72c7f4b-fbd4-496b-b66c-800629e3a75e","Type":"ContainerStarted","Data":"77caa584d12996444a0a56583a6ff6768106c3ca3c38417435c6fc869eefed2e"}
Sep 16 14:04:49.160564 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.160542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zgdjq" event={"ID":"9758160b-dbeb-4a30-9f3e-f2e3f491aea5","Type":"ContainerStarted","Data":"41ba9aa2e969d60249b1638f9630f26bbec90897143faa9d5e28e376359cc39c"}
Sep 16 14:04:49.163738 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.163701    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-txpr4" podStartSLOduration=1.538722794 podStartE2EDuration="19.16368607s" totalImagesPullingTime="17.624963276s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:49.163102152 +0000 UTC m=+19.651112228" watchObservedRunningTime="2026-09-16 14:04:49.16368607 +0000 UTC m=+19.651696147"
Sep 16 14:04:49.190936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.190846    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zgdjq" podStartSLOduration=1.698961441 podStartE2EDuration="19.190831007s" totalImagesPullingTime="17.491869566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:49.190628293 +0000 UTC m=+19.678638368" watchObservedRunningTime="2026-09-16 14:04:49.190831007 +0000 UTC m=+19.678841125"
Sep 16 14:04:49.238863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.238809    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5jpt6" podStartSLOduration=5.879792644 podStartE2EDuration="19.238789067s" totalImagesPullingTime="13.358996423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:49.238525343 +0000 UTC m=+19.726535421" watchObservedRunningTime="2026-09-16 14:04:49.238789067 +0000 UTC m=+19.726799146"
Sep 16 14:04:49.256046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.255996    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pv8ks" podStartSLOduration=1.794969537 podStartE2EDuration="19.25597995s" totalImagesPullingTime="17.461010413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:49.255521469 +0000 UTC m=+19.743531545" watchObservedRunningTime="2026-09-16 14:04:49.25597995 +0000 UTC m=+19.743990026"
Sep 16 14:04:49.312642 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:49.312586    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-34.ec2.internal" podStartSLOduration=19.312570932 podStartE2EDuration="19.312570932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:49.312224145 +0000 UTC m=+19.800234222" watchObservedRunningTime="2026-09-16 14:04:49.312570932 +0000 UTC m=+19.800581008"
Sep 16 14:04:50.104118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.104093    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 14:04:50.163539 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.163500    2580 generic.go:356] "Generic (PLEG): container finished" podID="13e1e09ae1d6a5cf42212bb85eac6f19" containerID="0cf9a896e08384490a95dd568c6a4a6b638f1a80e4042035346f3cdcab9e62eb" exitCode=0
Sep 16 14:04:50.163707 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.163592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal" event={"ID":"13e1e09ae1d6a5cf42212bb85eac6f19","Type":"ContainerDied","Data":"0cf9a896e08384490a95dd568c6a4a6b638f1a80e4042035346f3cdcab9e62eb"}
Sep 16 14:04:50.166692 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.166669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"c6175e47cdeb30f129bb2445c9d1ebd2080c80fdd60afba8ec15b30769eee423"}
Sep 16 14:04:50.166797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.166736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"4c61338e0d0619afe31bdf9c23a6d440b8b50df4f89fa750e413a5f38b86ef73"}
Sep 16 14:04:50.166797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.166775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"7e84299ded7dafa53ef735f78cc336cf81fe3e7e0664ca225516879faa98c45e"}
Sep 16 14:04:50.166909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.166813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"6a49322c4e7703f9c236ffdc9a4f02f885932ba3a4d4e1188b33a0fcab92ba0f"}
Sep 16 14:04:50.168260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.168240    2580 generic.go:356] "Generic (PLEG): container finished" podID="206c7143-17a7-46b9-93b9-c1509d26ccc1" containerID="c2684cee1b5b68658c458fd8800d45f20f4c5deba4dde5aa124344f0b848d580" exitCode=0
Sep 16 14:04:50.168430 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.168321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerDied","Data":"c2684cee1b5b68658c458fd8800d45f20f4c5deba4dde5aa124344f0b848d580"}
Sep 16 14:04:50.169905 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.169881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mns9d" event={"ID":"a01415cc-8ef7-4e60-8deb-0b314dd4fe3d","Type":"ContainerStarted","Data":"475c969d78332842b0f4e60f90919800953fcab354a0a36d17b506549942c76e"}
Sep 16 14:04:50.171761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.171739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7" event={"ID":"82cd6309-31f0-45bf-913d-869248730c15","Type":"ContainerStarted","Data":"3d66b7eb83e41b616bf3d5534a1ea057908da56c7a4cd20221e6335235e86efe"}
Sep 16 14:04:50.211965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.211930    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mns9d" podStartSLOduration=1.946848951 podStartE2EDuration="20.211917253s" totalImagesPullingTime="18.265068302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-16 14:04:50.211665835 +0000 UTC m=+20.699675921" watchObservedRunningTime="2026-09-16 14:04:50.211917253 +0000 UTC m=+20.699927328"
Sep 16 14:04:50.910273 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.910164    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T14:04:50.104114541Z","UUID":"95ed634c-5987-4602-98b4-1d51a866d7a5","Handler":null,"Name":"","Endpoint":""}
Sep 16 14:04:50.911868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.911840    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 14:04:50.911988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.911878    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 14:04:51.000427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:50.999790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:51.000427 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:50.999922    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:51.000427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.000359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:51.000667 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:51.000464    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:51.000667 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.000496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:51.000667 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:51.000592    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:51.174868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.174801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p7frh" event={"ID":"cf58899c-5f52-4b1c-82a9-2adc1b62999d","Type":"ContainerStarted","Data":"76fc60ac33803a5daacfcae4296ab0fb9ced6527a24c528ca41a9c89aabe455d"}
Sep 16 14:04:51.176764 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.176725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7" event={"ID":"82cd6309-31f0-45bf-913d-869248730c15","Type":"ContainerStarted","Data":"200a9ef0dd49e66b912874f11e22624ac454ef1b8db0998f6eab71d008feefa1"}
Sep 16 14:04:51.178661 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.178635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal" event={"ID":"13e1e09ae1d6a5cf42212bb85eac6f19","Type":"ContainerStarted","Data":"cec31a79b5bd88e90f1d29321ded2eaae1479077a739ef1bf15d825c97762e72"}
Sep 16 14:04:51.190369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.190329    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p7frh" podStartSLOduration=3.674318779 podStartE2EDuration="21.190314678s" totalImagesPullingTime="17.515995899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:51.18939946 +0000 UTC m=+21.677409535" watchObservedRunningTime="2026-09-16 14:04:51.190314678 +0000 UTC m=+21.678324793"
Sep 16 14:04:51.207229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.207177    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thkw7" podStartSLOduration=1.909214554 podStartE2EDuration="21.207156416s" totalImagesPullingTime="19.297941862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:51.20663819 +0000 UTC m=+21.694648277" watchObservedRunningTime="2026-09-16 14:04:51.207156416 +0000 UTC m=+21.695166501"
Sep 16 14:04:51.222027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:51.221979    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-34.ec2.internal" podStartSLOduration=21.221960938 podStartE2EDuration="21.221960938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:51.221839007 +0000 UTC m=+21.709849083" watchObservedRunningTime="2026-09-16 14:04:51.221960938 +0000 UTC m=+21.709971019"
Sep 16 14:04:52.185154 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:52.184816    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"0479cd9cf500423951b88922ccc35d1e1fc6409b0378f57b8a866c12b367a8d9"}
Sep 16 14:04:52.549270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:52.549234    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:52.549808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:52.549788    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:52.671570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:52.671533    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:52.672174 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:52.672121    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5jpt6"
Sep 16 14:04:52.999897 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:52.999863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:52.999897 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:52.999899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:53.000157 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:52.999997    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:53.000157 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:53.000112    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:53.000274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:53.000194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:53.000325 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:53.000286    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:55.000396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.000226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:55.000755 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.000235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:55.000755 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:55.000472    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:55.000755 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:55.000556    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:55.000755 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.000235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:55.000755 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:55.000637    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:55.195206 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.195159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" event={"ID":"c8c153ad-02b2-4d61-956b-2bded0db29a6","Type":"ContainerStarted","Data":"4044db29428d9708a50541177a8da79a8c17ae80381a48ec7d38c654b1bb6676"}
Sep 16 14:04:55.195533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.195509    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:55.197185 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.197160    2580 generic.go:356] "Generic (PLEG): container finished" podID="206c7143-17a7-46b9-93b9-c1509d26ccc1" containerID="47d3c2ce1d14d3d6470948218d8cdd2e74175397f1f76b1930b1ebb4916c7069" exitCode=0
Sep 16 14:04:55.197275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.197217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerDied","Data":"47d3c2ce1d14d3d6470948218d8cdd2e74175397f1f76b1930b1ebb4916c7069"}
Sep 16 14:04:55.211682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.211654    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:55.229230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:55.229179    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4" podStartSLOduration=7.717311481 podStartE2EDuration="25.229161937s" totalImagesPullingTime="17.511850456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:55.228079149 +0000 UTC m=+25.716089224" watchObservedRunningTime="2026-09-16 14:04:55.229161937 +0000 UTC m=+25.717172006"
Sep 16 14:04:56.202205 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.202166    2580 generic.go:356] "Generic (PLEG): container finished" podID="206c7143-17a7-46b9-93b9-c1509d26ccc1" containerID="7dcc59d182093c2a628f73519c0ca7f42c3852fd813a3c5cf63293d6b67974fd" exitCode=0
Sep 16 14:04:56.203027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.202256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerDied","Data":"7dcc59d182093c2a628f73519c0ca7f42c3852fd813a3c5cf63293d6b67974fd"}
Sep 16 14:04:56.203027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.202688    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:56.203027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.202765    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:56.218306 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.218280    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:04:56.368079 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.368049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pq29s"]
Sep 16 14:04:56.368243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.368209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:56.368330 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:56.368312    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:56.372030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.371962    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d96cv"]
Sep 16 14:04:56.372128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.372054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:56.372190 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:56.372122    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:56.374949 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.374927    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-px6lq"]
Sep 16 14:04:56.375027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:56.375008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:56.375093 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:56.375074    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:57.999796 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:57.999765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:04:58.000203 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:57.999808    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:04:58.000203 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:57.999766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:04:58.000203 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:57.999884    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:04:58.000203 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:57.999959    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:04:58.000203 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:04:58.000066    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:04:59.028494 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:59.028317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:04:59.126008 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:04:59.125964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:00.000555 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:00.000522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:05:00.000717 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:00.000607    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d96cv" podUID="603ccdba-2511-4336-9480-1fa9490508d8"
Sep 16 14:05:00.000717 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:00.000640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:05:00.000717 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:00.000675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:05:00.000824 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:00.000772    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px6lq" podUID="dd12c3f1-620d-4992-bb8b-68c1293ab5ab"
Sep 16 14:05:00.000874 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:00.000854    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pq29s" podUID="24c0e5cb-4a9e-4377-8e92-61022a699d47"
Sep 16 14:05:00.214288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:00.214257    2580 generic.go:356] "Generic (PLEG): container finished" podID="206c7143-17a7-46b9-93b9-c1509d26ccc1" containerID="d3921d19783eebc998600c1c5012b8f502ef7ae645a7b02861c9a5a87cbd51f4" exitCode=0
Sep 16 14:05:00.214732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:00.214300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerDied","Data":"d3921d19783eebc998600c1c5012b8f502ef7ae645a7b02861c9a5a87cbd51f4"}
Sep 16 14:05:00.487766 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:00.487656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:00.588780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:00.588718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:01.815616 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.815583    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-34.ec2.internal" event="NodeReady"
Sep 16 14:05:01.816029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.815729    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 14:05:01.863325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.863288    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-785d65c679-x9sjs"]
Sep 16 14:05:01.887844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.887807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-785d65c679-x9sjs"]
Sep 16 14:05:01.888133 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.888111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.889709 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.889686    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f9rpw"]
Sep 16 14:05:01.891786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.891451    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 14:05:01.891786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.891451    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 14:05:01.891786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.891652    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 14:05:01.891786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.891702    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nfdsg\""
Sep 16 14:05:01.899229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.899204    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 14:05:01.905070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.905048    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ghpvl"]
Sep 16 14:05:01.905189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.905153    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:01.913047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.912888    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 14:05:01.913047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.912892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 14:05:01.913047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.912924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ss8nr\""
Sep 16 14:05:01.923038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.923018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ghpvl"]
Sep 16 14:05:01.923165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.923053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f9rpw"]
Sep 16 14:05:01.923165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.923156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:01.926915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.926896    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jrwbv\""
Sep 16 14:05:01.927131 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.927109    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 14:05:01.927232 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.927171    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 14:05:01.927450 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.927434    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 14:05:01.982831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.982795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-tmp-dir\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:01.982831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.982833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-registry-certificates\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.982866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pgtk\" (UniqueName: \"kubernetes.io/projected/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-kube-api-access-4pgtk\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:01.983029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.982911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-image-registry-private-configuration\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.982943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.982968    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-trusted-ca\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.982994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc44n\" (UniqueName: \"kubernetes.io/projected/95b03975-d212-411b-92dd-db3740693f1f-kube-api-access-zc44n\") pod \"ingress-canary-ghpvl\" (UID: \"95b03975-d212-411b-92dd-db3740693f1f\") " pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:01.983247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.983046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sb42\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-kube-api-access-7sb42\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.983066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-config-volume\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:01.983247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.983093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-bound-sa-token\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.983171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-installation-pull-secrets\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.983229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ee42a76-1ee0-4272-87c4-df314623e616-ca-trust-extracted\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:01.983445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.983251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:01.983445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.983266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b03975-d212-411b-92dd-db3740693f1f-cert\") pod \"ingress-canary-ghpvl\" (UID: \"95b03975-d212-411b-92dd-db3740693f1f\") " pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:01.999932 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.999874    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:05:01.999932 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.999899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:05:01.999932 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:01.999905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:05:02.003123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.003101    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j74pt\""
Sep 16 14:05:02.003504 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.003487    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 14:05:02.003590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.003486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 14:05:02.003659 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.003588    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 14:05:02.003742 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.003726    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df78v\""
Sep 16 14:05:02.003806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.003792    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 14:05:02.084328 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sb42\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-kube-api-access-7sb42\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084328 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-config-volume\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-bound-sa-token\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-installation-pull-secrets\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ee42a76-1ee0-4272-87c4-df314623e616-ca-trust-extracted\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b03975-d212-411b-92dd-db3740693f1f-cert\") pod \"ingress-canary-ghpvl\" (UID: \"95b03975-d212-411b-92dd-db3740693f1f\") " pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-tmp-dir\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-registry-certificates\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pgtk\" (UniqueName: \"kubernetes.io/projected/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-kube-api-access-4pgtk\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-image-registry-private-configuration\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-trusted-ca\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.084924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc44n\" (UniqueName: \"kubernetes.io/projected/95b03975-d212-411b-92dd-db3740693f1f-kube-api-access-zc44n\") pod \"ingress-canary-ghpvl\" (UID: \"95b03975-d212-411b-92dd-db3740693f1f\") " pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:02.084924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-tmp-dir\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.084924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.084866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-config-volume\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.085306 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.085280    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:02.085413 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.085341    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls podName:7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:02.585323504 +0000 UTC m=+33.073333579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls") pod "dns-default-f9rpw" (UID: "7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:02.085413 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.085349    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:05:02.085413 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.085361    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785d65c679-x9sjs: secret "image-registry-tls" not found
Sep 16 14:05:02.085413 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.085393    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls podName:0ee42a76-1ee0-4272-87c4-df314623e616 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:02.585382925 +0000 UTC m=+33.073392992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls") pod "image-registry-785d65c679-x9sjs" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616") : secret "image-registry-tls" not found
Sep 16 14:05:02.085620 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.085280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ee42a76-1ee0-4272-87c4-df314623e616-ca-trust-extracted\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.089415 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.089396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-installation-pull-secrets\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.089500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.089450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-image-registry-private-configuration\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.089500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.089481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b03975-d212-411b-92dd-db3740693f1f-cert\") pod \"ingress-canary-ghpvl\" (UID: \"95b03975-d212-411b-92dd-db3740693f1f\") " pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:02.096628 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.096603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc44n\" (UniqueName: \"kubernetes.io/projected/95b03975-d212-411b-92dd-db3740693f1f-kube-api-access-zc44n\") pod \"ingress-canary-ghpvl\" (UID: \"95b03975-d212-411b-92dd-db3740693f1f\") " pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:02.096772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.096749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-registry-certificates\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.097122 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.097103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sb42\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-kube-api-access-7sb42\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.097280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.097247    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-trusted-ca\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.097459 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.097438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-bound-sa-token\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.097905 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.097864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pgtk\" (UniqueName: \"kubernetes.io/projected/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-kube-api-access-4pgtk\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.231372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.231340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ghpvl"
Sep 16 14:05:02.544660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.544404    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ghpvl"]
Sep 16 14:05:02.549246 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:02.549215    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b03975_d212_411b_92dd_db3740693f1f.slice/crio-67c55e9049588add6b380c73b61a73a8a43b85b92d48bb55b9f63f6e1f71b98c WatchSource:0}: Error finding container 67c55e9049588add6b380c73b61a73a8a43b85b92d48bb55b9f63f6e1f71b98c: Status 404 returned error can't find the container with id 67c55e9049588add6b380c73b61a73a8a43b85b92d48bb55b9f63f6e1f71b98c
Sep 16 14:05:02.587608 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.587577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:02.587719 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.587638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:02.587719 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.587709    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:02.587822 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.587752    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:05:02.587822 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.587765    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785d65c679-x9sjs: secret "image-registry-tls" not found
Sep 16 14:05:02.587822 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.587777    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls podName:7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:03.587758564 +0000 UTC m=+34.075768641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls") pod "dns-default-f9rpw" (UID: "7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:02.587822 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.587814    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls podName:0ee42a76-1ee0-4272-87c4-df314623e616 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:03.587797852 +0000 UTC m=+34.075807907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls") pod "image-registry-785d65c679-x9sjs" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616") : secret "image-registry-tls" not found
Sep 16 14:05:02.688799 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.688764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:05:02.688982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.688818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:05:02.688982 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.688917    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 14:05:02.689089 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:02.688985    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs podName:24c0e5cb-4a9e-4377-8e92-61022a699d47 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:34.688964974 +0000 UTC m=+65.176975052 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs") pod "network-metrics-daemon-pq29s" (UID: "24c0e5cb-4a9e-4377-8e92-61022a699d47") : secret "metrics-daemon-secret" not found
Sep 16 14:05:02.692781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.692615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd12c3f1-620d-4992-bb8b-68c1293ab5ab-original-pull-secret\") pod \"global-pull-secret-syncer-px6lq\" (UID: \"dd12c3f1-620d-4992-bb8b-68c1293ab5ab\") " pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:05:02.789483 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.789444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:05:02.792123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.792098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvlrg\" (UniqueName: \"kubernetes.io/projected/603ccdba-2511-4336-9480-1fa9490508d8-kube-api-access-cvlrg\") pod \"network-check-target-d96cv\" (UID: \"603ccdba-2511-4336-9480-1fa9490508d8\") " pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:05:02.911699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.911613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:05:02.917850 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:02.917816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px6lq"
Sep 16 14:05:03.215530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:03.215503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d96cv"]
Sep 16 14:05:03.222403 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:03.222372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ghpvl" event={"ID":"95b03975-d212-411b-92dd-db3740693f1f","Type":"ContainerStarted","Data":"67c55e9049588add6b380c73b61a73a8a43b85b92d48bb55b9f63f6e1f71b98c"}
Sep 16 14:05:03.245783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:03.245759    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-px6lq"]
Sep 16 14:05:03.597114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:03.597028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:03.597114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:03.597100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:03.597308 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:03.597217    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:03.597308 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:03.597233    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:05:03.597308 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:03.597253    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785d65c679-x9sjs: secret "image-registry-tls" not found
Sep 16 14:05:03.597308 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:03.597294    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls podName:7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:05.59727381 +0000 UTC m=+36.085283888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls") pod "dns-default-f9rpw" (UID: "7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:03.597308 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:03.597311    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls podName:0ee42a76-1ee0-4272-87c4-df314623e616 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:05.597305046 +0000 UTC m=+36.085315100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls") pod "image-registry-785d65c679-x9sjs" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616") : secret "image-registry-tls" not found
Sep 16 14:05:04.870884 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:04.870850    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603ccdba_2511_4336_9480_1fa9490508d8.slice/crio-3f8224953ba7346a7d103b20ddd7ec7c24d8680ad7056eb4fff13ab3c48abe09 WatchSource:0}: Error finding container 3f8224953ba7346a7d103b20ddd7ec7c24d8680ad7056eb4fff13ab3c48abe09: Status 404 returned error can't find the container with id 3f8224953ba7346a7d103b20ddd7ec7c24d8680ad7056eb4fff13ab3c48abe09
Sep 16 14:05:05.229678 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:05.229644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d96cv" event={"ID":"603ccdba-2511-4336-9480-1fa9490508d8","Type":"ContainerStarted","Data":"3f8224953ba7346a7d103b20ddd7ec7c24d8680ad7056eb4fff13ab3c48abe09"}
Sep 16 14:05:05.307681 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:05.307637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:05.611156 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:05.611069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:05.611156 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:05.611151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:05.611320 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:05.611236    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:05.611320 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:05.611242    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:05:05.611320 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:05.611260    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785d65c679-x9sjs: secret "image-registry-tls" not found
Sep 16 14:05:05.611320 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:05.611286    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls podName:7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:09.611273567 +0000 UTC m=+40.099283621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls") pod "dns-default-f9rpw" (UID: "7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:05.611320 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:05.611305    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls podName:0ee42a76-1ee0-4272-87c4-df314623e616 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:09.611292287 +0000 UTC m=+40.099302341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls") pod "image-registry-785d65c679-x9sjs" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616") : secret "image-registry-tls" not found
Sep 16 14:05:05.620915 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:05.620889    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd12c3f1_620d_4992_bb8b_68c1293ab5ab.slice/crio-dcee73d3508537217fd0ae4eef39475f7093d391403c00fb0f412319def77d6d WatchSource:0}: Error finding container dcee73d3508537217fd0ae4eef39475f7093d391403c00fb0f412319def77d6d: Status 404 returned error can't find the container with id dcee73d3508537217fd0ae4eef39475f7093d391403c00fb0f412319def77d6d
Sep 16 14:05:06.241047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:06.240684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerStarted","Data":"7d42493f72d7b411e4dc7e6f8e0e5cf37d47a3f4a17f869a2728ca78d9446270"}
Sep 16 14:05:06.244310 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:06.244279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ghpvl" event={"ID":"95b03975-d212-411b-92dd-db3740693f1f","Type":"ContainerStarted","Data":"c7011ddbd938ed0b971d16ec6dc57d51adf554ecaf9385c1aee1398e4ba45bfb"}
Sep 16 14:05:06.246742 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:06.246682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-px6lq" event={"ID":"dd12c3f1-620d-4992-bb8b-68c1293ab5ab","Type":"ContainerStarted","Data":"dcee73d3508537217fd0ae4eef39475f7093d391403c00fb0f412319def77d6d"}
Sep 16 14:05:06.285857 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:06.285372    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ghpvl" podStartSLOduration=1.8264902250000001 podStartE2EDuration="5.28532806s" totalImagesPullingTime="3.458837835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:06.284595842 +0000 UTC m=+36.772605919" watchObservedRunningTime="2026-09-16 14:05:06.28532806 +0000 UTC m=+36.773338142"
Sep 16 14:05:06.665483 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:06.665036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:07.252782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.252644    2580 generic.go:356] "Generic (PLEG): container finished" podID="206c7143-17a7-46b9-93b9-c1509d26ccc1" containerID="7d42493f72d7b411e4dc7e6f8e0e5cf37d47a3f4a17f869a2728ca78d9446270" exitCode=0
Sep 16 14:05:07.252782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.252686    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerDied","Data":"7d42493f72d7b411e4dc7e6f8e0e5cf37d47a3f4a17f869a2728ca78d9446270"}
Sep 16 14:05:07.554103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.553589    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"]
Sep 16 14:05:07.563450 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.563425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"
Sep 16 14:05:07.567000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.566976    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 16 14:05:07.567106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.566976    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 16 14:05:07.567106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.566980    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-slh6l\""
Sep 16 14:05:07.572351 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.572292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"]
Sep 16 14:05:07.733933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.733897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdm66\" (UniqueName: \"kubernetes.io/projected/1b511e3c-f247-4dbd-bfbe-c8256bcae36c-kube-api-access-xdm66\") pod \"migrator-8b4d697c5-xqbm5\" (UID: \"1b511e3c-f247-4dbd-bfbe-c8256bcae36c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"
Sep 16 14:05:07.834945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.834874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdm66\" (UniqueName: \"kubernetes.io/projected/1b511e3c-f247-4dbd-bfbe-c8256bcae36c-kube-api-access-xdm66\") pod \"migrator-8b4d697c5-xqbm5\" (UID: \"1b511e3c-f247-4dbd-bfbe-c8256bcae36c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"
Sep 16 14:05:07.855653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.855621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdm66\" (UniqueName: \"kubernetes.io/projected/1b511e3c-f247-4dbd-bfbe-c8256bcae36c-kube-api-access-xdm66\") pod \"migrator-8b4d697c5-xqbm5\" (UID: \"1b511e3c-f247-4dbd-bfbe-c8256bcae36c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"
Sep 16 14:05:07.874682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.874651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"
Sep 16 14:05:07.978416 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:07.978094    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fccsf"]
Sep 16 14:05:08.011786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.011751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fccsf"]
Sep 16 14:05:08.012003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.011979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.014908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.014811    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 14:05:08.015285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.015265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 14:05:08.015502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.015485    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 14:05:08.015577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.015554    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 14:05:08.015676 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.015499    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzz2w\""
Sep 16 14:05:08.137790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.137706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.137955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.137792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hbwt\" (UniqueName: \"kubernetes.io/projected/eba65688-a180-4429-8372-fd32ccd5532d-kube-api-access-4hbwt\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.137955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.137831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eba65688-a180-4429-8372-fd32ccd5532d-crio-socket\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.137955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.137859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.137955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.137903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eba65688-a180-4429-8372-fd32ccd5532d-data-volume\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.162283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.162220    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5"]
Sep 16 14:05:08.238667 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.238628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.238826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.238705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hbwt\" (UniqueName: \"kubernetes.io/projected/eba65688-a180-4429-8372-fd32ccd5532d-kube-api-access-4hbwt\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.238826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.238744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eba65688-a180-4429-8372-fd32ccd5532d-crio-socket\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.238826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.238771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.238826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.238814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eba65688-a180-4429-8372-fd32ccd5532d-data-volume\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.239067 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.239047    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:08.239125 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.239100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eba65688-a180-4429-8372-fd32ccd5532d-crio-socket\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.239261 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.239126    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:08.739106839 +0000 UTC m=+39.227116910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:08.239261 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.239181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eba65688-a180-4429-8372-fd32ccd5532d-data-volume\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.239261 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.239200    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.239261 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.239262    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:08.73924311 +0000 UTC m=+39.227253172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.253472 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.253440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hbwt\" (UniqueName: \"kubernetes.io/projected/eba65688-a180-4429-8372-fd32ccd5532d-kube-api-access-4hbwt\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.260059 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.259440    2580 generic.go:356] "Generic (PLEG): container finished" podID="206c7143-17a7-46b9-93b9-c1509d26ccc1" containerID="b5c438a42b80daf1e1e416013445b04fbdeffb12a9194d82a41d46d50b3420b0" exitCode=0
Sep 16 14:05:08.260059 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.259483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerDied","Data":"b5c438a42b80daf1e1e416013445b04fbdeffb12a9194d82a41d46d50b3420b0"}
Sep 16 14:05:08.742411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.742366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.742585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:08.742463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:08.742585 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.742569    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.742693 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.742636    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:09.742617721 +0000 UTC m=+40.230627783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.742693 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.742569    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:08.742801 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:08.742744    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:09.742721584 +0000 UTC m=+40.230731661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:09.087698 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:09.087636    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b511e3c_f247_4dbd_bfbe_c8256bcae36c.slice/crio-2f37c3813273519bb82539c89dbe6df9d3939108f7f331de1b2b93e32c4f8879 WatchSource:0}: Error finding container 2f37c3813273519bb82539c89dbe6df9d3939108f7f331de1b2b93e32c4f8879: Status 404 returned error can't find the container with id 2f37c3813273519bb82539c89dbe6df9d3939108f7f331de1b2b93e32c4f8879
Sep 16 14:05:09.263425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.263385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5" event={"ID":"1b511e3c-f247-4dbd-bfbe-c8256bcae36c","Type":"ContainerStarted","Data":"2f37c3813273519bb82539c89dbe6df9d3939108f7f331de1b2b93e32c4f8879"}
Sep 16 14:05:09.457345 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.457162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:09.541442 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.541405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:09.634209 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.634176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:09.650152 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.650110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:09.650288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.650203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:09.650288 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.650263    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:09.650399 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.650312    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:05:09.650399 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.650323    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785d65c679-x9sjs: secret "image-registry-tls" not found
Sep 16 14:05:09.650399 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.650332    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls podName:7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:17.65031424 +0000 UTC m=+48.138324314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls") pod "dns-default-f9rpw" (UID: "7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:09.650399 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.650364    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls podName:0ee42a76-1ee0-4272-87c4-df314623e616 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:17.650350065 +0000 UTC m=+48.138360137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls") pod "image-registry-785d65c679-x9sjs" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616") : secret "image-registry-tls" not found
Sep 16 14:05:09.750859 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.750829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:09.751030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:09.750903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:09.751030 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.751013    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:09.751129 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.751093    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:11.751074565 +0000 UTC m=+42.239084636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:09.751129 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.751018    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:09.751223 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:09.751180    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:11.751167503 +0000 UTC m=+42.239177557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:10.458982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:10.458950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:10.525660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:10.525627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:10.703554 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:10.703525    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pv8ks_d72c7f4b-fbd4-496b-b66c-800629e3a75e/dns-node-resolver/0.log"
Sep 16 14:05:11.270070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.270038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d96cv" event={"ID":"603ccdba-2511-4336-9480-1fa9490508d8","Type":"ContainerStarted","Data":"203a3b976fc38bfb215570d0cc74bfc3d949a8f5d55f8a94d8939a5a787cbb75"}
Sep 16 14:05:11.270258 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.270163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:05:11.271603 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.271573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-px6lq" event={"ID":"dd12c3f1-620d-4992-bb8b-68c1293ab5ab","Type":"ContainerStarted","Data":"2a020a6f9a4a315650eb7b987e2bf52ba98bdeff7af25a07b1dc05aa1496f15d"}
Sep 16 14:05:11.274935 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.274915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fx94s" event={"ID":"206c7143-17a7-46b9-93b9-c1509d26ccc1","Type":"ContainerStarted","Data":"d1def43c4bb4d847c8f3ec17967835d19ab38e64dee9ce7320d9ea28a036178c"}
Sep 16 14:05:11.289801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.289758    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d96cv" podStartSLOduration=35.263385415 podStartE2EDuration="41.289747514s" totalImagesPullingTime="6.026362099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:11.289094045 +0000 UTC m=+41.777104121" watchObservedRunningTime="2026-09-16 14:05:11.289747514 +0000 UTC m=+41.777757589"
Sep 16 14:05:11.307152 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.307091    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-px6lq" podStartSLOduration=36.366379975 podStartE2EDuration="41.307078764s" totalImagesPullingTime="4.940698789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:11.306309456 +0000 UTC m=+41.794319531" watchObservedRunningTime="2026-09-16 14:05:11.307078764 +0000 UTC m=+41.795088839"
Sep 16 14:05:11.338723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.337564    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fx94s" podStartSLOduration=10.12848021 podStartE2EDuration="41.337540763s" totalImagesPullingTime="31.209060553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:11.336035856 +0000 UTC m=+41.824045931" watchObservedRunningTime="2026-09-16 14:05:11.337540763 +0000 UTC m=+41.825550843"
Sep 16 14:05:11.397022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.396986    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:11.510559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.510532    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mns9d_a01415cc-8ef7-4e60-8deb-0b314dd4fe3d/node-ca/0.log"
Sep 16 14:05:11.768208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.768163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:11.768443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:11.768258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:11.768443 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:11.768339    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:11.768443 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:11.768407    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:15.768390932 +0000 UTC m=+46.256400991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:11.768443 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:11.768339    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:11.768603 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:11.768470    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:15.768456223 +0000 UTC m=+46.256466295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:12.279669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:12.279630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5" event={"ID":"1b511e3c-f247-4dbd-bfbe-c8256bcae36c","Type":"ContainerStarted","Data":"f77a031cc739adbb8776952dff5368ace0f623a3cdc3fa408b95a1e717488335"}
Sep 16 14:05:12.279802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:12.279682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5" event={"ID":"1b511e3c-f247-4dbd-bfbe-c8256bcae36c","Type":"ContainerStarted","Data":"d52470a198fa3eda5d6415864cab85b1544924c63dd303071180fa5a31daaff3"}
Sep 16 14:05:12.292469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:12.292412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:12.298888 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:12.298819    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xqbm5" podStartSLOduration=2.34190269 podStartE2EDuration="5.298803605s" totalImagesPullingTime="2.956900915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:12.297112274 +0000 UTC m=+42.785122349" watchObservedRunningTime="2026-09-16 14:05:12.298803605 +0000 UTC m=+42.786813737"
Sep 16 14:05:12.302916 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:12.302892    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ghpvl_95b03975-d212-411b-92dd-db3740693f1f/serve-healthcheck-canary/0.log"
Sep 16 14:05:12.903497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:12.903459    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-xqbm5_1b511e3c-f247-4dbd-bfbe-c8256bcae36c/migrator/0.log"
Sep 16 14:05:13.103929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:13.103899    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-xqbm5_1b511e3c-f247-4dbd-bfbe-c8256bcae36c/graceful-termination/0.log"
Sep 16 14:05:14.282774 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:14.282732    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:14.819665 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:14.819631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:14.921196 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:14.921161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:15.045736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:15.045705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:15.151510 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:15.151429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:15.248384 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:15.248345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:15.348085 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:15.348049    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:15.554265 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:15.554227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:15.797312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:15.797280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:15.797519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:15.797342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:15.797519 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:15.797405    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:15.797519 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:15.797454    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:15.797519 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:15.797463    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:23.797449002 +0000 UTC m=+54.285459056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:15.797667 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:15.797557    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:23.797537341 +0000 UTC m=+54.285547407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:16.708171 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:16.708121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:16.911911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:16.911887    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:17.125338 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:17.125309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:17.320639 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:17.320609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:17.509845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:17.509810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:17.712753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:17.712710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") pod \"image-registry-785d65c679-x9sjs\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") " pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:17.713122 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:17.712804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:17.713122 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:17.712876    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:05:17.713122 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:17.712894    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785d65c679-x9sjs: secret "image-registry-tls" not found
Sep 16 14:05:17.713122 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:17.712900    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:17.713122 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:17.712962    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls podName:0ee42a76-1ee0-4272-87c4-df314623e616 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:33.712943941 +0000 UTC m=+64.200954009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls") pod "image-registry-785d65c679-x9sjs" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616") : secret "image-registry-tls" not found
Sep 16 14:05:17.713122 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:17.712979    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls podName:7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:33.712972077 +0000 UTC m=+64.200982132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls") pod "dns-default-f9rpw" (UID: "7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:18.221394 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:18.221356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:18.535769 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:18.535685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:18.716454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:18.716425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:19.330812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:19.330779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:19.427359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:19.427330    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:19.529152 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:19.529111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:20.711562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:20.711526    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:21.942224 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:21.942187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:22.042555 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:22.042517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:22.444016 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:22.443978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:22.720742 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:22.720707    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:22.914801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:22.914768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:23.115267 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:23.115183    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:23.437034 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:23.436951    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:23.537734 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:23.537695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:23.857194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:23.857153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:23.857390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:23.857239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:23.857390 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:23.857381    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:23.857466 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:23.857451    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls podName:eba65688-a180-4429-8372-fd32ccd5532d nodeName:}" failed. No retries permitted until 2026-09-16 14:05:39.857434094 +0000 UTC m=+70.345444149 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fccsf" (UID: "eba65688-a180-4429-8372-fd32ccd5532d") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:23.857655 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:23.857638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eba65688-a180-4429-8372-fd32ccd5532d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:24.197492 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:24.197407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:24.262365 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:24.262331    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:24.507933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:24.507897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:24.909946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:24.909875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:25.151215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:25.151180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:25.311547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:25.311505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:25.508467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:25.508434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:25.754169 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:25.754128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:28.217875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:28.217851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwwx4"
Sep 16 14:05:29.573737 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:29.573705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:29.671991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:29.671950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:29.880130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:29.880053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:30.271414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.271382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:30.314465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.314439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"]
Sep 16 14:05:30.316825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.316809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.320423 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.320400    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 16 14:05:30.320538 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.320400    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 14:05:30.320538 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.320444    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 14:05:30.320538 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.320470    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 14:05:30.334576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.334554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"]
Sep 16 14:05:30.348438 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.346853    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-785d65c679-x9sjs"]
Sep 16 14:05:30.348438 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:30.347270    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-785d65c679-x9sjs" podUID="0ee42a76-1ee0-4272-87c4-df314623e616"
Sep 16 14:05:30.402963 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.402934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.403110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.402965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-tmp\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.403110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.403015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnt8w\" (UniqueName: \"kubernetes.io/projected/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-kube-api-access-xnt8w\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.415766 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.415739    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-b2677"]
Sep 16 14:05:30.417460 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.417440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-b2677"
Sep 16 14:05:30.420265 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.420235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:30.428026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.428007    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 14:05:30.428124 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.428032    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 14:05:30.428124 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.428093    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6g2k5\""
Sep 16 14:05:30.428679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.428663    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75b95bb9b8-mprfv"]
Sep 16 14:05:30.430441 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.430428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.444343 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.444325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-b2677"]
Sep 16 14:05:30.477997 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.477972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75b95bb9b8-mprfv"]
Sep 16 14:05:30.503518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d913ed1-3adb-4f64-96a2-d70d06616ae1-image-registry-private-configuration\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.503618 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d913ed1-3adb-4f64-96a2-d70d06616ae1-installation-pull-secrets\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.503618 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-bound-sa-token\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.503618 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnt8w\" (UniqueName: \"kubernetes.io/projected/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-kube-api-access-xnt8w\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.503760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d913ed1-3adb-4f64-96a2-d70d06616ae1-trusted-ca\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.503760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5mcq\" (UniqueName: \"kubernetes.io/projected/47c5ce39-c04d-4810-ad73-63c0d59db14a-kube-api-access-h5mcq\") pod \"downloads-76f99584f4-b2677\" (UID: \"47c5ce39-c04d-4810-ad73-63c0d59db14a\") " pod="openshift-console/downloads-76f99584f4-b2677"
Sep 16 14:05:30.503760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d913ed1-3adb-4f64-96a2-d70d06616ae1-ca-trust-extracted\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.503760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d913ed1-3adb-4f64-96a2-d70d06616ae1-registry-certificates\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.503876 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-registry-tls\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.503876 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-tmp\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.503876 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.503876 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.503839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vww7\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-kube-api-access-7vww7\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.504123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.504108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-tmp\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.506112 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.506088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.530273 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.530225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnt8w\" (UniqueName: \"kubernetes.io/projected/3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4-kube-api-access-xnt8w\") pod \"klusterlet-addon-workmgr-6bdfbd5f47-n292r\" (UID: \"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.604123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d913ed1-3adb-4f64-96a2-d70d06616ae1-trusted-ca\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5mcq\" (UniqueName: \"kubernetes.io/projected/47c5ce39-c04d-4810-ad73-63c0d59db14a-kube-api-access-h5mcq\") pod \"downloads-76f99584f4-b2677\" (UID: \"47c5ce39-c04d-4810-ad73-63c0d59db14a\") " pod="openshift-console/downloads-76f99584f4-b2677"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d913ed1-3adb-4f64-96a2-d70d06616ae1-ca-trust-extracted\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d913ed1-3adb-4f64-96a2-d70d06616ae1-registry-certificates\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-registry-tls\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vww7\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-kube-api-access-7vww7\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d913ed1-3adb-4f64-96a2-d70d06616ae1-image-registry-private-configuration\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d913ed1-3adb-4f64-96a2-d70d06616ae1-installation-pull-secrets\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-bound-sa-token\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.604635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.604546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d913ed1-3adb-4f64-96a2-d70d06616ae1-ca-trust-extracted\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.605222 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.605042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d913ed1-3adb-4f64-96a2-d70d06616ae1-registry-certificates\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.605222 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.605080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d913ed1-3adb-4f64-96a2-d70d06616ae1-trusted-ca\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.606767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.606745    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d913ed1-3adb-4f64-96a2-d70d06616ae1-installation-pull-secrets\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.606889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.606870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d913ed1-3adb-4f64-96a2-d70d06616ae1-image-registry-private-configuration\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.606961 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.606942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-registry-tls\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.616208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.616184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5mcq\" (UniqueName: \"kubernetes.io/projected/47c5ce39-c04d-4810-ad73-63c0d59db14a-kube-api-access-h5mcq\") pod \"downloads-76f99584f4-b2677\" (UID: \"47c5ce39-c04d-4810-ad73-63c0d59db14a\") " pod="openshift-console/downloads-76f99584f4-b2677"
Sep 16 14:05:30.616721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.616703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-bound-sa-token\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.616760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.616693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vww7\" (UniqueName: \"kubernetes.io/projected/5d913ed1-3adb-4f64-96a2-d70d06616ae1-kube-api-access-7vww7\") pod \"image-registry-75b95bb9b8-mprfv\" (UID: \"5d913ed1-3adb-4f64-96a2-d70d06616ae1\") " pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.625598 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.625579    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:30.726253 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.726221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-b2677"
Sep 16 14:05:30.739688 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.739660    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nfdsg\""
Sep 16 14:05:30.747884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.747862    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:30.893989 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.893930    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"]
Sep 16 14:05:30.895970 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:30.895943    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b492cc7_e0f1_41ea_8c13_28ee71f9b4b4.slice/crio-246ad8fdfe2d842c9b46caf1a07257ee1d72b005daff7aa907d8ad74cf269718 WatchSource:0}: Error finding container 246ad8fdfe2d842c9b46caf1a07257ee1d72b005daff7aa907d8ad74cf269718: Status 404 returned error can't find the container with id 246ad8fdfe2d842c9b46caf1a07257ee1d72b005daff7aa907d8ad74cf269718
Sep 16 14:05:30.992541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:30.992514    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-b2677"]
Sep 16 14:05:30.995431 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:30.995405    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c5ce39_c04d_4810_ad73_63c0d59db14a.slice/crio-22b919899ae3b77ea2b1d630aef7e348f157743372ffb78a32ae865ab2fb05e0 WatchSource:0}: Error finding container 22b919899ae3b77ea2b1d630aef7e348f157743372ffb78a32ae865ab2fb05e0: Status 404 returned error can't find the container with id 22b919899ae3b77ea2b1d630aef7e348f157743372ffb78a32ae865ab2fb05e0
Sep 16 14:05:31.027799 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.027769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75b95bb9b8-mprfv"]
Sep 16 14:05:31.031108 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:31.031069    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d913ed1_3adb_4f64_96a2_d70d06616ae1.slice/crio-6d604727f06eb6dfb8b0156367d76397a9b8f491778109a4727228b5be99e784 WatchSource:0}: Error finding container 6d604727f06eb6dfb8b0156367d76397a9b8f491778109a4727228b5be99e784: Status 404 returned error can't find the container with id 6d604727f06eb6dfb8b0156367d76397a9b8f491778109a4727228b5be99e784
Sep 16 14:05:31.330163 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.330070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv" event={"ID":"5d913ed1-3adb-4f64-96a2-d70d06616ae1","Type":"ContainerStarted","Data":"954813d1de649b5211e7c43eb30092650a9c0e70d918a1e382af1c72813a0b3f"}
Sep 16 14:05:31.330163 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.330121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv" event={"ID":"5d913ed1-3adb-4f64-96a2-d70d06616ae1","Type":"ContainerStarted","Data":"6d604727f06eb6dfb8b0156367d76397a9b8f491778109a4727228b5be99e784"}
Sep 16 14:05:31.330388 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.330256    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:31.331301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.331278    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-b2677" event={"ID":"47c5ce39-c04d-4810-ad73-63c0d59db14a","Type":"ContainerStarted","Data":"22b919899ae3b77ea2b1d630aef7e348f157743372ffb78a32ae865ab2fb05e0"}
Sep 16 14:05:31.332483 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.332454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r" event={"ID":"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4","Type":"ContainerStarted","Data":"246ad8fdfe2d842c9b46caf1a07257ee1d72b005daff7aa907d8ad74cf269718"}
Sep 16 14:05:31.332591 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.332471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:31.338416 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.338394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:31.351262 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.351221    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv" podStartSLOduration=1.351209512 podStartE2EDuration="1.351209512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:31.350342628 +0000 UTC m=+61.838352705" watchObservedRunningTime="2026-09-16 14:05:31.351209512 +0000 UTC m=+61.839219587"
Sep 16 14:05:31.410680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.410646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-image-registry-private-configuration\") pod \"0ee42a76-1ee0-4272-87c4-df314623e616\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") "
Sep 16 14:05:31.410846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.410697    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-registry-certificates\") pod \"0ee42a76-1ee0-4272-87c4-df314623e616\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") "
Sep 16 14:05:31.410846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.410737    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sb42\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-kube-api-access-7sb42\") pod \"0ee42a76-1ee0-4272-87c4-df314623e616\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") "
Sep 16 14:05:31.410846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.410761    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-trusted-ca\") pod \"0ee42a76-1ee0-4272-87c4-df314623e616\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") "
Sep 16 14:05:31.410846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.410800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ee42a76-1ee0-4272-87c4-df314623e616-ca-trust-extracted\") pod \"0ee42a76-1ee0-4272-87c4-df314623e616\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") "
Sep 16 14:05:31.410846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.410831    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-bound-sa-token\") pod \"0ee42a76-1ee0-4272-87c4-df314623e616\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") "
Sep 16 14:05:31.411082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.410860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-installation-pull-secrets\") pod \"0ee42a76-1ee0-4272-87c4-df314623e616\" (UID: \"0ee42a76-1ee0-4272-87c4-df314623e616\") "
Sep 16 14:05:31.411158 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.411120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0ee42a76-1ee0-4272-87c4-df314623e616" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:05:31.411216 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.411203    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-registry-certificates\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:31.411269 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.411213    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee42a76-1ee0-4272-87c4-df314623e616-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0ee42a76-1ee0-4272-87c4-df314623e616" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:05:31.411867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.411838    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0ee42a76-1ee0-4272-87c4-df314623e616" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:05:31.413410 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.413376    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0ee42a76-1ee0-4272-87c4-df314623e616" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:05:31.413410 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.413386    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-kube-api-access-7sb42" (OuterVolumeSpecName: "kube-api-access-7sb42") pod "0ee42a76-1ee0-4272-87c4-df314623e616" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616"). InnerVolumeSpecName "kube-api-access-7sb42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:05:31.413577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.413466    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0ee42a76-1ee0-4272-87c4-df314623e616" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:05:31.413861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.413838    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0ee42a76-1ee0-4272-87c4-df314623e616" (UID: "0ee42a76-1ee0-4272-87c4-df314623e616"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:05:31.515898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.512998    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-image-registry-private-configuration\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:31.515898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.513031    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sb42\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-kube-api-access-7sb42\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:31.515898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.513053    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ee42a76-1ee0-4272-87c4-df314623e616-trusted-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:31.515898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.513068    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ee42a76-1ee0-4272-87c4-df314623e616-ca-trust-extracted\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:31.515898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.513082    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-bound-sa-token\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:31.515898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.513096    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ee42a76-1ee0-4272-87c4-df314623e616-installation-pull-secrets\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:31.796292 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.796252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:31.864426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:31.864383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:32.336805 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:32.336413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785d65c679-x9sjs"
Sep 16 14:05:32.369239 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:32.369202    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-785d65c679-x9sjs"]
Sep 16 14:05:32.372478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:32.372451    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-785d65c679-x9sjs"]
Sep 16 14:05:32.420646 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:32.420412    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ee42a76-1ee0-4272-87c4-df314623e616-registry-tls\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:05:32.871207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:32.871174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:32.979460 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:32.979425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:33.732033 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:33.731976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:33.735011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:33.734981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0-metrics-tls\") pod \"dns-default-f9rpw\" (UID: \"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0\") " pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:33.972594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:33.972519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:34.004502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.004422    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ee42a76-1ee0-4272-87c4-df314623e616" path="/var/lib/kubelet/pods/0ee42a76-1ee0-4272-87c4-df314623e616/volumes"
Sep 16 14:05:34.016885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.016681    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ss8nr\""
Sep 16 14:05:34.024366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.024337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:34.072182 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.072076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:34.097126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.096982    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"]
Sep 16 14:05:34.100090 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.100063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:34.102897 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.102872    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-45k7b\""
Sep 16 14:05:34.103024 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.102945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 16 14:05:34.107739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.107718    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"]
Sep 16 14:05:34.135870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.135840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7lr9m\" (UID: \"280b5349-c479-4d5d-96f2-de6262f6c471\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:34.190367 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.190333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:34.237264 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.237231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7lr9m\" (UID: \"280b5349-c479-4d5d-96f2-de6262f6c471\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:34.237413 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:34.237347    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 14:05:34.237481 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:34.237429    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates podName:280b5349-c479-4d5d-96f2-de6262f6c471 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:34.737410163 +0000 UTC m=+65.225420239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-7lr9m" (UID: "280b5349-c479-4d5d-96f2-de6262f6c471") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 14:05:34.656239 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.656200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:34.742621 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.742588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:05:34.742786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.742651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7lr9m\" (UID: \"280b5349-c479-4d5d-96f2-de6262f6c471\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:34.742786 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:34.742751    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 14:05:34.742929 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:34.742832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates podName:280b5349-c479-4d5d-96f2-de6262f6c471 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:35.742811904 +0000 UTC m=+66.230821958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-7lr9m" (UID: "280b5349-c479-4d5d-96f2-de6262f6c471") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 14:05:34.745090 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:34.745068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c0e5cb-4a9e-4377-8e92-61022a699d47-metrics-certs\") pod \"network-metrics-daemon-pq29s\" (UID: \"24c0e5cb-4a9e-4377-8e92-61022a699d47\") " pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:05:35.025936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.025902    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df78v\""
Sep 16 14:05:35.034042 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.034018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pq29s"
Sep 16 14:05:35.057294 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.057268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f9rpw"]
Sep 16 14:05:35.060949 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:35.060908    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0e0b1e_3871_47b7_8ddb_1b2716dcc0e0.slice/crio-4b97ab7947d2653359382bcde81801a6cc6547a47631ec87a4052c862d00fa18 WatchSource:0}: Error finding container 4b97ab7947d2653359382bcde81801a6cc6547a47631ec87a4052c862d00fa18: Status 404 returned error can't find the container with id 4b97ab7947d2653359382bcde81801a6cc6547a47631ec87a4052c862d00fa18
Sep 16 14:05:35.314219 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.314191    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pq29s"]
Sep 16 14:05:35.316868 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:35.316836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c0e5cb_4a9e_4377_8e92_61022a699d47.slice/crio-c3ed28f87e25f83106331f43dfe2148b3c7a6e6b27d3b41785652e1c12f8b287 WatchSource:0}: Error finding container c3ed28f87e25f83106331f43dfe2148b3c7a6e6b27d3b41785652e1c12f8b287: Status 404 returned error can't find the container with id c3ed28f87e25f83106331f43dfe2148b3c7a6e6b27d3b41785652e1c12f8b287
Sep 16 14:05:35.350329 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.350292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r" event={"ID":"3b492cc7-e0f1-41ea-8c13-28ee71f9b4b4","Type":"ContainerStarted","Data":"91592c5d536d3d81fb628517d35c8341181f98b0353b3db77e46f4424a294671"}
Sep 16 14:05:35.350510 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.350487    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:35.351969 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.351943    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pq29s" event={"ID":"24c0e5cb-4a9e-4377-8e92-61022a699d47","Type":"ContainerStarted","Data":"c3ed28f87e25f83106331f43dfe2148b3c7a6e6b27d3b41785652e1c12f8b287"}
Sep 16 14:05:35.352428 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.352403    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r"
Sep 16 14:05:35.353539 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.353515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f9rpw" event={"ID":"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0","Type":"ContainerStarted","Data":"4b97ab7947d2653359382bcde81801a6cc6547a47631ec87a4052c862d00fa18"}
Sep 16 14:05:35.368347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.368297    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bdfbd5f47-n292r" podStartSLOduration=1.407826788 podStartE2EDuration="5.368277995s" totalImagesPullingTime="3.960451207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:35.367260987 +0000 UTC m=+65.855271065" watchObservedRunningTime="2026-09-16 14:05:35.368277995 +0000 UTC m=+65.856288072"
Sep 16 14:05:35.752566 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.752532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7lr9m\" (UID: \"280b5349-c479-4d5d-96f2-de6262f6c471\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:35.755521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.755491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/280b5349-c479-4d5d-96f2-de6262f6c471-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7lr9m\" (UID: \"280b5349-c479-4d5d-96f2-de6262f6c471\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:35.912936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:35.912898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:36.024153 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:36.024061    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:36.105171 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:36.105097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:36.195006 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:36.194976    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"]
Sep 16 14:05:36.198686 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:36.198410    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280b5349_c479_4d5d_96f2_de6262f6c471.slice/crio-e2d5d49ed953b4ed6deef984f33704ffad66956fd9773fbd19767faab5683a3c WatchSource:0}: Error finding container e2d5d49ed953b4ed6deef984f33704ffad66956fd9773fbd19767faab5683a3c: Status 404 returned error can't find the container with id e2d5d49ed953b4ed6deef984f33704ffad66956fd9773fbd19767faab5683a3c
Sep 16 14:05:36.359903 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:36.359824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m" event={"ID":"280b5349-c479-4d5d-96f2-de6262f6c471","Type":"ContainerStarted","Data":"e2d5d49ed953b4ed6deef984f33704ffad66956fd9773fbd19767faab5683a3c"}
Sep 16 14:05:37.364993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:37.364960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f9rpw" event={"ID":"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0","Type":"ContainerStarted","Data":"6fb109de56573ca596b4816a0be67e260b6a44187676444060cd0f29998f2b72"}
Sep 16 14:05:37.365453 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:37.365020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f9rpw" event={"ID":"7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0","Type":"ContainerStarted","Data":"72d6ea3f446988a8f0d7f435a44fb40db0ab54b2e274def74922ecd74f1925e2"}
Sep 16 14:05:37.365519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:37.365484    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:37.383278 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:37.383225    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f9rpw" podStartSLOduration=34.549310174 podStartE2EDuration="36.383205955s" totalImagesPullingTime="1.833895781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:37.381778986 +0000 UTC m=+67.869789064" watchObservedRunningTime="2026-09-16 14:05:37.383205955 +0000 UTC m=+67.871216031"
Sep 16 14:05:38.370861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:38.370713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m" event={"ID":"280b5349-c479-4d5d-96f2-de6262f6c471","Type":"ContainerStarted","Data":"c026b3149435b9dd4c103c6d235e581a7aed80809a940b5c6513db83617c0fa5"}
Sep 16 14:05:38.370861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:38.370763    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:38.376524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:38.376494    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m"
Sep 16 14:05:38.388254 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:38.388204    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7lr9m" podStartSLOduration=2.564001248 podStartE2EDuration="4.38818493s" totalImagesPullingTime="1.824183682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:38.387624852 +0000 UTC m=+68.875634928" watchObservedRunningTime="2026-09-16 14:05:38.38818493 +0000 UTC m=+68.876195007"
Sep 16 14:05:39.129900 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.129859    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:39.171574 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.171539    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"]
Sep 16 14:05:39.189752 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.189726    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"]
Sep 16 14:05:39.189906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.189844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.192845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.192642    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 16 14:05:39.192845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.192707    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 14:05:39.192845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.192707    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 14:05:39.193075 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.192929    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 16 14:05:39.193840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.193818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lqfgt\""
Sep 16 14:05:39.193981 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.193894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 14:05:39.216145 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.216108    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:39.284660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.284612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d2725af-2346-43a9-b973-eff34547355a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.284779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.284665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d2725af-2346-43a9-b973-eff34547355a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.284779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.284689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5sb5\" (UniqueName: \"kubernetes.io/projected/8d2725af-2346-43a9-b973-eff34547355a-kube-api-access-s5sb5\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.284779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.284764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d2725af-2346-43a9-b973-eff34547355a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.289798 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.289772    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:39.385777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.385718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d2725af-2346-43a9-b973-eff34547355a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.385777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.385754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d2725af-2346-43a9-b973-eff34547355a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.385777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.385776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5sb5\" (UniqueName: \"kubernetes.io/projected/8d2725af-2346-43a9-b973-eff34547355a-kube-api-access-s5sb5\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.386291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.385827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d2725af-2346-43a9-b973-eff34547355a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.386458 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.386416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d2725af-2346-43a9-b973-eff34547355a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.388429 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.388405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d2725af-2346-43a9-b973-eff34547355a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.388759 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.388734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d2725af-2346-43a9-b973-eff34547355a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.396397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.396359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5sb5\" (UniqueName: \"kubernetes.io/projected/8d2725af-2346-43a9-b973-eff34547355a-kube-api-access-s5sb5\") pod \"prometheus-operator-68bcd7f987-6q9pq\" (UID: \"8d2725af-2346-43a9-b973-eff34547355a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.501123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.501099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"
Sep 16 14:05:39.811728 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.811694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq"]
Sep 16 14:05:39.814699 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:39.814664    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2725af_2346_43a9_b973_eff34547355a.slice/crio-01dffc604ff04c0f2bbfd154303b86f14c6bd6ae59c47af0439aa1ced1ae89db WatchSource:0}: Error finding container 01dffc604ff04c0f2bbfd154303b86f14c6bd6ae59c47af0439aa1ced1ae89db: Status 404 returned error can't find the container with id 01dffc604ff04c0f2bbfd154303b86f14c6bd6ae59c47af0439aa1ced1ae89db
Sep 16 14:05:39.889469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.889429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:39.891749 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:39.891731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eba65688-a180-4429-8372-fd32ccd5532d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fccsf\" (UID: \"eba65688-a180-4429-8372-fd32ccd5532d\") " pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:40.127866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.127747    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzz2w\""
Sep 16 14:05:40.136124 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.136101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fccsf"
Sep 16 14:05:40.380239 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.380128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pq29s" event={"ID":"24c0e5cb-4a9e-4377-8e92-61022a699d47","Type":"ContainerStarted","Data":"a2d40206f463f762c91da80911b468035add5001fb822d3f2a612245a8bc0f69"}
Sep 16 14:05:40.380239 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.380217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pq29s" event={"ID":"24c0e5cb-4a9e-4377-8e92-61022a699d47","Type":"ContainerStarted","Data":"a859c45e2481ed82d047e56bd5f7d8181c8f100e04b2d0d77cebac1edc948b05"}
Sep 16 14:05:40.381612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.381584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq" event={"ID":"8d2725af-2346-43a9-b973-eff34547355a","Type":"ContainerStarted","Data":"01dffc604ff04c0f2bbfd154303b86f14c6bd6ae59c47af0439aa1ced1ae89db"}
Sep 16 14:05:40.402070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.402019    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pq29s" podStartSLOduration=66.46816538 podStartE2EDuration="1m10.40200137s" totalImagesPullingTime="3.93383599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:40.400532298 +0000 UTC m=+70.888542375" watchObservedRunningTime="2026-09-16 14:05:40.40200137 +0000 UTC m=+70.890011445"
Sep 16 14:05:40.450129 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.450094    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fccsf"]
Sep 16 14:05:40.453895 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:40.453862    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba65688_a180_4429_8372_fd32ccd5532d.slice/crio-30524daff06433d3821f9220f3b1760bb7aa33b4de5f5fe2f85cbaa24586612a WatchSource:0}: Error finding container 30524daff06433d3821f9220f3b1760bb7aa33b4de5f5fe2f85cbaa24586612a: Status 404 returned error can't find the container with id 30524daff06433d3821f9220f3b1760bb7aa33b4de5f5fe2f85cbaa24586612a
Sep 16 14:05:40.493654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.493622    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:40.592528 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.592447    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:40.693906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:40.693825    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:41.389294 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:41.389123    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fccsf" event={"ID":"eba65688-a180-4429-8372-fd32ccd5532d","Type":"ContainerStarted","Data":"a0be3cf32dcfc0cac03ed2821e9c5dc7752b538449c76f6dc9b590ba9528adb7"}
Sep 16 14:05:41.389294 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:41.389248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fccsf" event={"ID":"eba65688-a180-4429-8372-fd32ccd5532d","Type":"ContainerStarted","Data":"30524daff06433d3821f9220f3b1760bb7aa33b4de5f5fe2f85cbaa24586612a"}
Sep 16 14:05:41.887603 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:41.887567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:41.988567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:41.988515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:42.089544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:42.089518    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:42.282394 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:42.282316    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d96cv"
Sep 16 14:05:42.394707 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:42.394667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq" event={"ID":"8d2725af-2346-43a9-b973-eff34547355a","Type":"ContainerStarted","Data":"103f3824806da669f583204fced16df8372c70f3e37c5d8320c74275408777e2"}
Sep 16 14:05:42.394870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:42.394747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq" event={"ID":"8d2725af-2346-43a9-b973-eff34547355a","Type":"ContainerStarted","Data":"21e1aa837e32a634229685eb0b3ee525b36f989c859e638b0ec7f5776984abac"}
Sep 16 14:05:42.396735 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:42.396706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fccsf" event={"ID":"eba65688-a180-4429-8372-fd32ccd5532d","Type":"ContainerStarted","Data":"b03e9df1cf2272d3cc54558f72d9a033955c5be4e60c2fca11bbcb7187c3ad8d"}
Sep 16 14:05:42.415194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:42.415127    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-6q9pq" podStartSLOduration=1.521843129 podStartE2EDuration="3.415112316s" totalImagesPullingTime="1.893269187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:42.414595976 +0000 UTC m=+72.902606053" watchObservedRunningTime="2026-09-16 14:05:42.415112316 +0000 UTC m=+72.903122385"
Sep 16 14:05:43.064840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.064806    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f9979f79d-ljclz"]
Sep 16 14:05:43.088826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.088792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9979f79d-ljclz"]
Sep 16 14:05:43.089007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.088935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.091972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.091934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w6sw4\""
Sep 16 14:05:43.091972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.091946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 14:05:43.092219 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.092043    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 14:05:43.092219 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.091946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 14:05:43.092381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.092361    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 14:05:43.092451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.092437    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 14:05:43.097806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.097788    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 14:05:43.214687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.214656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-oauth-serving-cert\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.214866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.214703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-oauth-config\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.214866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.214728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-service-ca\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.214866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.214853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-serving-cert\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.214994 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.214892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-config\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.214994 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.214921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-trusted-ca-bundle\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.214994 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.214941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztslt\" (UniqueName: \"kubernetes.io/projected/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-kube-api-access-ztslt\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.315871 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.315776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-serving-cert\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.315871 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.315841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-config\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.315871 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.315868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-trusted-ca-bundle\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.316213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.315888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztslt\" (UniqueName: \"kubernetes.io/projected/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-kube-api-access-ztslt\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.316213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.315940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-oauth-serving-cert\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.316213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.315975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-oauth-config\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.316213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.315997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-service-ca\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.316782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.316682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-service-ca\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.316782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.316682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-config\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.317328 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.317290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-oauth-serving-cert\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.317444 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.317323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-trusted-ca-bundle\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.318965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.318942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-oauth-config\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.319127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.319101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-serving-cert\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.324790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.324766    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztslt\" (UniqueName: \"kubernetes.io/projected/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-kube-api-access-ztslt\") pod \"console-6f9979f79d-ljclz\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") " pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:43.400568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:43.400537    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:05:44.557710 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.557676    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"]
Sep 16 14:05:44.573958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.573831    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"]
Sep 16 14:05:44.574133 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.573975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.578581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.578552    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 14:05:44.579097 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.579075    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-s9dl5\""
Sep 16 14:05:44.579701 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.579678    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 14:05:44.582986 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.582956    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p6cgd"]
Sep 16 14:05:44.605418 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.605394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.608048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.608030    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dpfk2\""
Sep 16 14:05:44.608300 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.608273    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 14:05:44.608512 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.608495    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 14:05:44.608740 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.608723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 14:05:44.628281 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.628249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.628406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.628303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.628406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.628346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.628406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.628385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fms4g\" (UniqueName: \"kubernetes.io/projected/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-kube-api-access-fms4g\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.729257 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.729455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.729455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj4sk\" (UniqueName: \"kubernetes.io/projected/a2ca0e60-1d2a-4845-a983-f6831571be9d-kube-api-access-tj4sk\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-textfile\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fms4g\" (UniqueName: \"kubernetes.io/projected/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-kube-api-access-fms4g\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.729725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-root\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-tls\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-wtmp\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ca0e60-1d2a-4845-a983-f6831571be9d-metrics-client-ca\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.729988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.729796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-sys\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.729988 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:44.729866    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 16 14:05:44.729988 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:44.729942    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-tls podName:1b190ccf-d681-4fd3-8e22-da2f1aafbea9 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:45.229922778 +0000 UTC m=+75.717932847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-6kgzp" (UID: "1b190ccf-d681-4fd3-8e22-da2f1aafbea9") : secret "openshift-state-metrics-tls" not found
Sep 16 14:05:44.730126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.730069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.732054 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.732027    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.740537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.740506    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fms4g\" (UniqueName: \"kubernetes.io/projected/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-kube-api-access-fms4g\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:44.831201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-sys\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj4sk\" (UniqueName: \"kubernetes.io/projected/a2ca0e60-1d2a-4845-a983-f6831571be9d-kube-api-access-tj4sk\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-textfile\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-sys\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-root\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-tls\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-wtmp\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-root\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ca0e60-1d2a-4845-a983-f6831571be9d-metrics-client-ca\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-textfile\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-wtmp\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.831984 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ca0e60-1d2a-4845-a983-f6831571be9d-metrics-client-ca\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.832036 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.831964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.833865 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.833841    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-tls\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.833987 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.833919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ca0e60-1d2a-4845-a983-f6831571be9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.840433 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.840408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj4sk\" (UniqueName: \"kubernetes.io/projected/a2ca0e60-1d2a-4845-a983-f6831571be9d-kube-api-access-tj4sk\") pod \"node-exporter-p6cgd\" (UID: \"a2ca0e60-1d2a-4845-a983-f6831571be9d\") " pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:44.915303 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:44.915274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p6cgd"
Sep 16 14:05:45.234304 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.234263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:45.236810 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.236781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b190ccf-d681-4fd3-8e22-da2f1aafbea9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6kgzp\" (UID: \"1b190ccf-d681-4fd3-8e22-da2f1aafbea9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:45.485842 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.485749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"
Sep 16 14:05:45.697927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.697892    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:05:45.702099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.702075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.704848 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.704612    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 14:05:45.704848 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.704663    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 14:05:45.704848 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.704729    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 14:05:45.705065 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.704927    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 14:05:45.705233 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.705207    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 14:05:45.705332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.705313    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 14:05:45.705385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.705313    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-45nr6\""
Sep 16 14:05:45.707276 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.707254    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 14:05:45.707699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.707668    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 14:05:45.707844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.707674    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 14:05:45.723828 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.723803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:05:45.738348 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738348 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-web-config\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-out\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm44n\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-kube-api-access-mm44n\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.738788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.738737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.839881 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.839845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.840060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.839907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.840060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.839923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-out\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.840060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.839941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.840060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.839965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.840060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.839996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.840060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.840023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm44n\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-kube-api-access-mm44n\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-web-config\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.841832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.842548 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:05:45.841947    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle podName:e84dbbfb-5f6a-4528-98ac-72410772b4cf nodeName:}" failed. No retries permitted until 2026-09-16 14:05:46.341927996 +0000 UTC m=+76.829938051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf") : configmap references non-existent config key: ca-bundle.crt
Sep 16 14:05:45.843648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.843602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.843888 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.843837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.844165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.844126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.844570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.844536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.844570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.844560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-web-config\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.844773 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.844689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.844773 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.844764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-out\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.845504 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.845481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.846544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.846522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:45.852806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:45.852781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm44n\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-kube-api-access-mm44n\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:46.255101 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.255056    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:46.346282 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.346249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:46.347280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.347248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:46.612875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.612792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:05:46.613152 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.613115    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8646694c88-ws45x"]
Sep 16 14:05:46.615770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.615614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.618895 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.618859    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-du6uur6oi1lcj\""
Sep 16 14:05:46.619032 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.618958    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ddz65\""
Sep 16 14:05:46.619032 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.619000    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 16 14:05:46.619203 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.619076    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 16 14:05:46.619276 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.619225    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 16 14:05:46.619334 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.619282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 16 14:05:46.619616 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.619598    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 16 14:05:46.627991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.627967    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8646694c88-ws45x"]
Sep 16 14:05:46.750896 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.750853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.751366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.750937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.751366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.750970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5424c0fc-020b-4532-8de4-361ef10b8771-metrics-client-ca\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.751366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.751009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-grpc-tls\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.751366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.751123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-tls\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.751366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.751176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.751366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.751244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gnnj\" (UniqueName: \"kubernetes.io/projected/5424c0fc-020b-4532-8de4-361ef10b8771-kube-api-access-4gnnj\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.751366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.751273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852167 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-tls\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852167 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852383 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gnnj\" (UniqueName: \"kubernetes.io/projected/5424c0fc-020b-4532-8de4-361ef10b8771-kube-api-access-4gnnj\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852383 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852383 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5424c0fc-020b-4532-8de4-361ef10b8771-metrics-client-ca\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.852525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.852485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-grpc-tls\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.853435 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.853369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5424c0fc-020b-4532-8de4-361ef10b8771-metrics-client-ca\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.855206 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.855130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.855343 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.855306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-tls\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.855561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.855537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.855672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.855650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.855801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.855774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.856732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.856712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5424c0fc-020b-4532-8de4-361ef10b8771-secret-grpc-tls\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.860425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.860408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gnnj\" (UniqueName: \"kubernetes.io/projected/5424c0fc-020b-4532-8de4-361ef10b8771-kube-api-access-4gnnj\") pod \"thanos-querier-8646694c88-ws45x\" (UID: \"5424c0fc-020b-4532-8de4-361ef10b8771\") " pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:46.926007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:46.925926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:48.377077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.377046    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f9rpw"
Sep 16 14:05:48.970057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.970019    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8494cd7db7-nl5x9"]
Sep 16 14:05:48.972913 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.972887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:48.975670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.975438    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 16 14:05:48.975670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.975501    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 16 14:05:48.975910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.975822    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-97hbz\""
Sep 16 14:05:48.976667 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.976646    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 16 14:05:48.976667 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.976660    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 16 14:05:48.976820 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.976734    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-194qb7v58krh1\""
Sep 16 14:05:48.985727 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:48.985707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8494cd7db7-nl5x9"]
Sep 16 14:05:49.076391 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.076348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6223c5b5-f143-4330-8e2c-303d614e486b-metrics-server-audit-profiles\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.076570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.076463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6223c5b5-f143-4330-8e2c-303d614e486b-audit-log\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.076570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.076541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-secret-metrics-server-tls\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.076672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.076583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-secret-metrics-server-client-certs\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.076672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.076609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-client-ca-bundle\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.076672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.076657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6223c5b5-f143-4330-8e2c-303d614e486b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.076784 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.076681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sjdc\" (UniqueName: \"kubernetes.io/projected/6223c5b5-f143-4330-8e2c-303d614e486b-kube-api-access-7sjdc\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.079895 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.079869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:49.121207 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:49.121069    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ca0e60_1d2a_4845_a983_f6831571be9d.slice/crio-f143f8155b26fa5f064f2cb55163295c88d41c28b3e52bad084781a3e1cad6b0 WatchSource:0}: Error finding container f143f8155b26fa5f064f2cb55163295c88d41c28b3e52bad084781a3e1cad6b0: Status 404 returned error can't find the container with id f143f8155b26fa5f064f2cb55163295c88d41c28b3e52bad084781a3e1cad6b0
Sep 16 14:05:49.177913 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.177801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-secret-metrics-server-client-certs\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.177913 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.177859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-client-ca-bundle\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.178057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.177913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6223c5b5-f143-4330-8e2c-303d614e486b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.178057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.177945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sjdc\" (UniqueName: \"kubernetes.io/projected/6223c5b5-f143-4330-8e2c-303d614e486b-kube-api-access-7sjdc\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.178057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.177985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6223c5b5-f143-4330-8e2c-303d614e486b-metrics-server-audit-profiles\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.178057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.178043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6223c5b5-f143-4330-8e2c-303d614e486b-audit-log\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.178299 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.178092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-secret-metrics-server-tls\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.183701 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.179305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6223c5b5-f143-4330-8e2c-303d614e486b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.183701 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.179699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6223c5b5-f143-4330-8e2c-303d614e486b-metrics-server-audit-profiles\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.193605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.188885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-client-ca-bundle\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.193605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.188964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6223c5b5-f143-4330-8e2c-303d614e486b-audit-log\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.193605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.190829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:49.201499 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.201473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sjdc\" (UniqueName: \"kubernetes.io/projected/6223c5b5-f143-4330-8e2c-303d614e486b-kube-api-access-7sjdc\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.202699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.202674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-secret-metrics-server-client-certs\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.206924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.206879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6223c5b5-f143-4330-8e2c-303d614e486b-secret-metrics-server-tls\") pod \"metrics-server-8494cd7db7-nl5x9\" (UID: \"6223c5b5-f143-4330-8e2c-303d614e486b\") " pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.285091 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.285062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:05:49.401370 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.401320    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp"]
Sep 16 14:05:49.406181 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:49.406128    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b190ccf_d681_4fd3_8e22_da2f1aafbea9.slice/crio-dddb7cd28c6e480ebb444797b4c249a555fd7e054f89d1e6e4b7c44ffabdd6fd WatchSource:0}: Error finding container dddb7cd28c6e480ebb444797b4c249a555fd7e054f89d1e6e4b7c44ffabdd6fd: Status 404 returned error can't find the container with id dddb7cd28c6e480ebb444797b4c249a555fd7e054f89d1e6e4b7c44ffabdd6fd
Sep 16 14:05:49.408657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.408555    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8646694c88-ws45x"]
Sep 16 14:05:49.420228 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.419441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:05:49.426159 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.426110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp" event={"ID":"1b190ccf-d681-4fd3-8e22-da2f1aafbea9","Type":"ContainerStarted","Data":"dddb7cd28c6e480ebb444797b4c249a555fd7e054f89d1e6e4b7c44ffabdd6fd"}
Sep 16 14:05:49.429272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.429173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" event={"ID":"5424c0fc-020b-4532-8de4-361ef10b8771","Type":"ContainerStarted","Data":"db4f01065940d19833873dcf7c0263255ce7fe8916f623bc0f65f8052f51b3f4"}
Sep 16 14:05:49.432052 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.431994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6cgd" event={"ID":"a2ca0e60-1d2a-4845-a983-f6831571be9d","Type":"ContainerStarted","Data":"f143f8155b26fa5f064f2cb55163295c88d41c28b3e52bad084781a3e1cad6b0"}
Sep 16 14:05:49.435635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.434771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-b2677" event={"ID":"47c5ce39-c04d-4810-ad73-63c0d59db14a","Type":"ContainerStarted","Data":"b11565299f2b87e70d6a26bff68ff5a7979bb89be2f8dd40cb7ed854e85b179e"}
Sep 16 14:05:49.435813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.435798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-b2677"
Sep 16 14:05:49.436937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.436833    2580 patch_prober.go:28] interesting pod/downloads-76f99584f4-b2677 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.12:8080/\": dial tcp 10.134.0.12:8080: connect: connection refused" start-of-body=
Sep 16 14:05:49.437033 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.436990    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-b2677" podUID="47c5ce39-c04d-4810-ad73-63c0d59db14a" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.12:8080/\": dial tcp 10.134.0.12:8080: connect: connection refused"
Sep 16 14:05:49.453510 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.453357    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-b2677" podStartSLOduration=1.234881201 podStartE2EDuration="19.453339192s" totalImagesPullingTime="18.218457991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:49.452104318 +0000 UTC m=+79.940114406" watchObservedRunningTime="2026-09-16 14:05:49.453339192 +0000 UTC m=+79.941349283"
Sep 16 14:05:49.455622 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.455599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9979f79d-ljclz"]
Sep 16 14:05:49.457741 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:49.457718    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cdd4d0_e47c_4a76_8609_3a910f1b25e5.slice/crio-c03577f9310a613ac80ae0eb6ae4209ee36587e12e87d9bb50f1280acf1963e2 WatchSource:0}: Error finding container c03577f9310a613ac80ae0eb6ae4209ee36587e12e87d9bb50f1280acf1963e2: Status 404 returned error can't find the container with id c03577f9310a613ac80ae0eb6ae4209ee36587e12e87d9bb50f1280acf1963e2
Sep 16 14:05:49.490147 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.490098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:49.575254 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.575227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8494cd7db7-nl5x9"]
Sep 16 14:05:49.579566 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.579426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:49.581229 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:49.581196    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6223c5b5_f143_4330_8e2c_303d614e486b.slice/crio-b3b131c10c0baa3a3e6f3f918315beb4cdd0a96d18ec3d4a0b3f4a6f9f0951ba WatchSource:0}: Error finding container b3b131c10c0baa3a3e6f3f918315beb4cdd0a96d18ec3d4a0b3f4a6f9f0951ba: Status 404 returned error can't find the container with id b3b131c10c0baa3a3e6f3f918315beb4cdd0a96d18ec3d4a0b3f4a6f9f0951ba
Sep 16 14:05:49.872942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:49.872847    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9979f79d-ljclz"]
Sep 16 14:05:50.446715 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.446679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fccsf" event={"ID":"eba65688-a180-4429-8372-fd32ccd5532d","Type":"ContainerStarted","Data":"5c084ffaccd34bac35a661ac6781f2c581f8aef6fe7553acf3dc39362310c346"}
Sep 16 14:05:50.450720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.450689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerStarted","Data":"5a9802985e9a3488674e956bbb548a59da7e3111d21c1cd75c14139a36906883"}
Sep 16 14:05:50.454332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.454304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9979f79d-ljclz" event={"ID":"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5","Type":"ContainerStarted","Data":"c03577f9310a613ac80ae0eb6ae4209ee36587e12e87d9bb50f1280acf1963e2"}
Sep 16 14:05:50.456340 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.456303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9" event={"ID":"6223c5b5-f143-4330-8e2c-303d614e486b","Type":"ContainerStarted","Data":"b3b131c10c0baa3a3e6f3f918315beb4cdd0a96d18ec3d4a0b3f4a6f9f0951ba"}
Sep 16 14:05:50.460814 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.460792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp" event={"ID":"1b190ccf-d681-4fd3-8e22-da2f1aafbea9","Type":"ContainerStarted","Data":"26836a67b3e81ef16c2d5eafe20c298426d9058880edcfb21cde01d8caca7503"}
Sep 16 14:05:50.460922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.460843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp" event={"ID":"1b190ccf-d681-4fd3-8e22-da2f1aafbea9","Type":"ContainerStarted","Data":"3bfc36fde05135de026f33a0c2a6746efffd769f0390c4b35b58a6dc0fa1c237"}
Sep 16 14:05:50.468055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.467928    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fccsf" podStartSLOduration=33.923514131 podStartE2EDuration="43.467910002s" totalImagesPullingTime="9.544395871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:05:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:50.467093166 +0000 UTC m=+80.955103243" watchObservedRunningTime="2026-09-16 14:05:50.467910002 +0000 UTC m=+80.955920078"
Sep 16 14:05:50.471404 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.471380    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-b2677"
Sep 16 14:05:50.770151 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.770098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:50.874985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.874030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:50.918822 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.917989    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:05:50.940052 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.939724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:05:50.940052 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.939892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:50.946067 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.943732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 14:05:50.946067 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.943945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 14:05:50.946067 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.944108    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 14:05:50.948770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.948109    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bknhso0dr3ehn\""
Sep 16 14:05:50.948770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.948462    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.957121    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.957604    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rf4np\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.958288    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.959655    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.960190    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.960603    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.960890    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 14:05:50.962172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.961848    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 14:05:50.971858 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.965624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:50.971858 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.966177    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 14:05:50.978435 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.978018    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 14:05:50.980535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.979965    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 14:05:50.999777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:50.999924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:50.999924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:50.999924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:50.999924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:50.999998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000024    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjzdh\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-kube-api-access-rjzdh\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.000647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.000350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.101895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.101950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.101992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjzdh\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-kube-api-access-rjzdh\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.102948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.102611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.103703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.103677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.104721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.103796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.104721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.104348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.104721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.104644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.110955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.110048    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.116583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.116558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.117081 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.117056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.119727 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.119666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.122867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.120562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.122867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.121074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.122867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.121356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjzdh\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-kube-api-access-rjzdh\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.122867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.122801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.123704 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.123660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.127965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.124130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.127965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.126814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.127965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.127925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.129623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.129574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.130928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.130702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.130928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.130886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.131116 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.131021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:51.274772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:51.274463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:05:52.344033 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:52.343998    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75b95bb9b8-mprfv"
Sep 16 14:05:52.484044 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:52.483801    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:52.548547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:52.548492    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:52.724020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:52.723969    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:52.888289 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:52.888242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:53.280395 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:53.280355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:53.382538 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:53.382500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:53.920916 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:53.920820    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:54.195542 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:54.195407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:54.294182 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:54.294127    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:54.393669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:54.393634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:55.386669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:55.386324    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:05:55.493794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:55.493756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp" event={"ID":"1b190ccf-d681-4fd3-8e22-da2f1aafbea9","Type":"ContainerStarted","Data":"9078afda414acb33674c2babbd510661d69bb8112cace65037affc1e4d4ad7a8"}
Sep 16 14:05:55.496558 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:55.496470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6cgd" event={"ID":"a2ca0e60-1d2a-4845-a983-f6831571be9d","Type":"ContainerStarted","Data":"96dc303fd05e2afcf26842d52be2b1237b52569bf7bfcc91e8ad085d8df576a8"}
Sep 16 14:05:55.498703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:55.498675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9" event={"ID":"6223c5b5-f143-4330-8e2c-303d614e486b","Type":"ContainerStarted","Data":"344fbee1544e72f1d2369daf580f31985ddce3cd40c7ee4e74592fa1de31dc8c"}
Sep 16 14:05:55.517047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:55.516964    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6kgzp" podStartSLOduration=5.987521659 podStartE2EDuration="11.516944801s" totalImagesPullingTime="5.529423142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:55.515702005 +0000 UTC m=+86.003712080" watchObservedRunningTime="2026-09-16 14:05:55.516944801 +0000 UTC m=+86.004955079"
Sep 16 14:05:55.567593 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:55.567530    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9" podStartSLOduration=2.055297359 podStartE2EDuration="7.567509086s" totalImagesPullingTime="5.512211727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:55.54139416 +0000 UTC m=+86.029404239" watchObservedRunningTime="2026-09-16 14:05:55.567509086 +0000 UTC m=+86.055519163"
Sep 16 14:05:55.705626 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:05:55.705541    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c5207a6_8974_4dec_82bb_c461a7bd7b2c.slice/crio-538d612a95956446b8998e2f79ea2ff45241e1bd5c0b21c5b493b77b55790a88 WatchSource:0}: Error finding container 538d612a95956446b8998e2f79ea2ff45241e1bd5c0b21c5b493b77b55790a88: Status 404 returned error can't find the container with id 538d612a95956446b8998e2f79ea2ff45241e1bd5c0b21c5b493b77b55790a88
Sep 16 14:05:55.800014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:55.799751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:56.505355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:56.505314    2580 generic.go:356] "Generic (PLEG): container finished" podID="a2ca0e60-1d2a-4845-a983-f6831571be9d" containerID="96dc303fd05e2afcf26842d52be2b1237b52569bf7bfcc91e8ad085d8df576a8" exitCode=0
Sep 16 14:05:56.505918 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:56.505404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6cgd" event={"ID":"a2ca0e60-1d2a-4845-a983-f6831571be9d","Type":"ContainerDied","Data":"96dc303fd05e2afcf26842d52be2b1237b52569bf7bfcc91e8ad085d8df576a8"}
Sep 16 14:05:56.506979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:56.506951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerStarted","Data":"538d612a95956446b8998e2f79ea2ff45241e1bd5c0b21c5b493b77b55790a88"}
Sep 16 14:05:57.042588 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.042557    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:57.157808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.157782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:05:57.512418 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.512386    2580 generic.go:356] "Generic (PLEG): container finished" podID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerID="6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd" exitCode=0
Sep 16 14:05:57.512884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.512461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd"}
Sep 16 14:05:57.514313 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.514263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9979f79d-ljclz" event={"ID":"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5","Type":"ContainerStarted","Data":"af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5"}
Sep 16 14:05:57.516744 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.516720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6cgd" event={"ID":"a2ca0e60-1d2a-4845-a983-f6831571be9d","Type":"ContainerStarted","Data":"e73cc9aa0c5b92d3a5050dae40ab8083903bb220a032bca949eea27b6ecaf2c4"}
Sep 16 14:05:57.516857 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.516769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p6cgd" event={"ID":"a2ca0e60-1d2a-4845-a983-f6831571be9d","Type":"ContainerStarted","Data":"f4129e20fb76163429f93ee169e8cdecc9d8883b8558d846d3e21cab957ce82d"}
Sep 16 14:05:57.518898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.518876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" event={"ID":"5424c0fc-020b-4532-8de4-361ef10b8771","Type":"ContainerStarted","Data":"f8091a499073a2e762d4fd8f39b008f351255b83ea8bcb2aefc6011df2cddb00"}
Sep 16 14:05:57.518980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.518932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" event={"ID":"5424c0fc-020b-4532-8de4-361ef10b8771","Type":"ContainerStarted","Data":"ab699b5b602d264f46e75f36a60166854db22207c2ed8da4ea55b605445572a9"}
Sep 16 14:05:57.518980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.518971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" event={"ID":"5424c0fc-020b-4532-8de4-361ef10b8771","Type":"ContainerStarted","Data":"aafc4efd1841a0e089413eabe8dbd88d70d928df2a41de1dcb7818b27f7fe2b2"}
Sep 16 14:05:57.520439 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.520416    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87" exitCode=0
Sep 16 14:05:57.520518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.520448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"}
Sep 16 14:05:57.590665 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.590576    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p6cgd" podStartSLOduration=7.562727156 podStartE2EDuration="13.590560291s" totalImagesPullingTime="5.954506489s" totalInitContainerRuntime="73.326646ms" isStatefulPod=true podCreationTimestamp="2026-09-16 14:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:57.589508671 +0000 UTC m=+88.077518748" watchObservedRunningTime="2026-09-16 14:05:57.590560291 +0000 UTC m=+88.078570367"
Sep 16 14:05:57.609544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:57.609498    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f9979f79d-ljclz" podStartSLOduration=7.257727012 podStartE2EDuration="14.609484907s" totalImagesPullingTime="7.351757895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:57.607740783 +0000 UTC m=+88.095750859" watchObservedRunningTime="2026-09-16 14:05:57.609484907 +0000 UTC m=+88.097494982"
Sep 16 14:05:59.530950 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:59.530913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" event={"ID":"5424c0fc-020b-4532-8de4-361ef10b8771","Type":"ContainerStarted","Data":"cbb5984083a0665820d30f69876cc54a2620d91f70c632d968a020d7171e2cfa"}
Sep 16 14:05:59.531380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:59.530998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" event={"ID":"5424c0fc-020b-4532-8de4-361ef10b8771","Type":"ContainerStarted","Data":"bf5f78155a37c53c941c6ba174554d900fb4faa2ce86984ad4013443afc02908"}
Sep 16 14:05:59.531380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:59.531024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" event={"ID":"5424c0fc-020b-4532-8de4-361ef10b8771","Type":"ContainerStarted","Data":"383deb7c35b4c7d2497454261ab63d17091f89738a85d42edeb77c9c48b071b1"}
Sep 16 14:05:59.531380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:59.531088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:05:59.555471 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:05:59.555422    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x" podStartSLOduration=4.526026367 podStartE2EDuration="13.555405425s" totalImagesPullingTime="9.029379058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:59.553495095 +0000 UTC m=+90.041505171" watchObservedRunningTime="2026-09-16 14:05:59.555405425 +0000 UTC m=+90.043415545"
Sep 16 14:06:00.163076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:00.163044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:00.252273 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:00.252234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:00.808459 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:00.808427    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:00.912912 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:00.912873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:01.012113 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:01.012078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:03.400879 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.400799    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:06:03.553300 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.553262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerStarted","Data":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"}
Sep 16 14:06:03.553477 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.553329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerStarted","Data":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"}
Sep 16 14:06:03.553477 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.553363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerStarted","Data":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"}
Sep 16 14:06:03.553477 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.553390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerStarted","Data":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"}
Sep 16 14:06:03.553477 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.553413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerStarted","Data":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"}
Sep 16 14:06:03.553477 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.553450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerStarted","Data":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"}
Sep 16 14:06:03.616217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:03.616163    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.375512535 podStartE2EDuration="13.616132177s" totalImagesPullingTime="6.228649523s" totalInitContainerRuntime="11.970119ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:06:03.611838272 +0000 UTC m=+94.099848348" watchObservedRunningTime="2026-09-16 14:06:03.616132177 +0000 UTC m=+94.104142305"
Sep 16 14:06:04.828768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:04.828720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:05.542877 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:05.542857    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8646694c88-ws45x"
Sep 16 14:06:05.562880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:05.562853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerStarted","Data":"6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175"}
Sep 16 14:06:06.275385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.275351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:06:06.499372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.499338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:06.570726 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.570639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerStarted","Data":"db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f"}
Sep 16 14:06:06.570726 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.570721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerStarted","Data":"41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2"}
Sep 16 14:06:06.570951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.570760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerStarted","Data":"ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a"}
Sep 16 14:06:06.570951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.570792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerStarted","Data":"3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c"}
Sep 16 14:06:06.570951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.570831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerStarted","Data":"e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f"}
Sep 16 14:06:06.609919 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:06.608261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:09.198355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:09.198321    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:09.286107 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:09.286078    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:06:09.286279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:09.286156    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:06:09.299912 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:09.299884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:22.533835 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.533770    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f9979f79d-ljclz" podUID="c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" containerName="console" containerID="cri-o://af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5" gracePeriod=15
Sep 16 14:06:22.784931 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.784878    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:06:22.802993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.802949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=22.407833114 podStartE2EDuration="37.802932039s" totalImagesPullingTime="15.359236189s" totalInitContainerRuntime="35.862736ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:06:06.608266487 +0000 UTC m=+97.096276590" watchObservedRunningTime="2026-09-16 14:06:22.802932039 +0000 UTC m=+113.290942114"
Sep 16 14:06:22.906547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.906509    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-serving-cert\") pod \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") "
Sep 16 14:06:22.906748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.906565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-config\") pod \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") "
Sep 16 14:06:22.906748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.906608    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-oauth-config\") pod \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") "
Sep 16 14:06:22.906748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.906643    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-trusted-ca-bundle\") pod \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") "
Sep 16 14:06:22.906748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.906666    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-oauth-serving-cert\") pod \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") "
Sep 16 14:06:22.906949 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.906778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-service-ca\") pod \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") "
Sep 16 14:06:22.906949 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.906806    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztslt\" (UniqueName: \"kubernetes.io/projected/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-kube-api-access-ztslt\") pod \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\" (UID: \"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5\") "
Sep 16 14:06:22.907130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.907099    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-config" (OuterVolumeSpecName: "console-config") pod "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" (UID: "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:22.907217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.907132    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" (UID: "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:22.907217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.907168    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" (UID: "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:22.907287 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.907272    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" (UID: "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:22.908886 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.908862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" (UID: "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:06:22.909022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.909003    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-kube-api-access-ztslt" (OuterVolumeSpecName: "kube-api-access-ztslt") pod "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" (UID: "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5"). InnerVolumeSpecName "kube-api-access-ztslt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:06:22.909178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:22.909157    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" (UID: "c4cdd4d0-e47c-4a76-8609-3a910f1b25e5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:06:23.007938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.007898    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-serving-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:06:23.007938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.007933    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-config\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:06:23.007938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.007942    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-console-oauth-config\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:06:23.007938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.007950    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-trusted-ca-bundle\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:06:23.008230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.007959    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-oauth-serving-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:06:23.008230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.007967    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:06:23.008230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.007975    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztslt\" (UniqueName: \"kubernetes.io/projected/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5-kube-api-access-ztslt\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:06:23.639153 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.639105    2580 generic.go:356] "Generic (PLEG): container finished" podID="c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" containerID="af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5" exitCode=0
Sep 16 14:06:23.639612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.639171    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9979f79d-ljclz"
Sep 16 14:06:23.639612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.639193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9979f79d-ljclz" event={"ID":"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5","Type":"ContainerDied","Data":"af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5"}
Sep 16 14:06:23.639612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.639254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9979f79d-ljclz" event={"ID":"c4cdd4d0-e47c-4a76-8609-3a910f1b25e5","Type":"ContainerDied","Data":"c03577f9310a613ac80ae0eb6ae4209ee36587e12e87d9bb50f1280acf1963e2"}
Sep 16 14:06:23.639612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.639290    2580 scope.go:117] "RemoveContainer" containerID="af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5"
Sep 16 14:06:23.647755 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.647733    2580 scope.go:117] "RemoveContainer" containerID="af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5"
Sep 16 14:06:23.648014 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:06:23.647996    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5\": container with ID starting with af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5 not found: ID does not exist" containerID="af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5"
Sep 16 14:06:23.648064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.648022    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5"} err="failed to get container status \"af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5\": rpc error: code = NotFound desc = could not find container \"af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5\": container with ID starting with af191cc79bf964c22b0ad49f173c9343cb5f474ffa75f05c702e46233bbd78c5 not found: ID does not exist"
Sep 16 14:06:23.659708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.659682    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9979f79d-ljclz"]
Sep 16 14:06:23.664731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:23.664710    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f9979f79d-ljclz"]
Sep 16 14:06:24.005960 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:24.005928    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" path="/var/lib/kubelet/pods/c4cdd4d0-e47c-4a76-8609-3a910f1b25e5/volumes"
Sep 16 14:06:24.195871 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:24.195838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:24.361710 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:24.361620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:28.438127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:28.438093    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ghpvl_95b03975-d212-411b-92dd-db3740693f1f/serve-healthcheck-canary/0.log"
Sep 16 14:06:29.291031 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:29.291002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:06:29.294777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:29.294753    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8494cd7db7-nl5x9"
Sep 16 14:06:31.774319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:31.774278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:31.893646 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:31.893613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:32.003841 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:32.003787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:06:51.275084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:51.275051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:06:51.293747 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:51.293726    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:06:51.766984 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:06:51.766952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:03.824514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:03.824479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:04.128089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.127997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:04.872003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.871965    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:07:04.872409 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.872276    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="alertmanager" containerID="cri-o://6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175" gracePeriod=120
Sep 16 14:07:04.872409 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.872321    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-metric" containerID="cri-o://41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2" gracePeriod=120
Sep 16 14:07:04.872409 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.872361    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy" containerID="cri-o://ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a" gracePeriod=120
Sep 16 14:07:04.872567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.872415    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="prom-label-proxy" containerID="cri-o://db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f" gracePeriod=120
Sep 16 14:07:04.872567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.872429    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="config-reloader" containerID="cri-o://e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f" gracePeriod=120
Sep 16 14:07:04.872567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:04.872367    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-web" containerID="cri-o://3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c" gracePeriod=120
Sep 16 14:07:05.816275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816244    2580 generic.go:356] "Generic (PLEG): container finished" podID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerID="db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f" exitCode=0
Sep 16 14:07:05.816275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816268    2580 generic.go:356] "Generic (PLEG): container finished" podID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerID="41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2" exitCode=0
Sep 16 14:07:05.816275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816274    2580 generic.go:356] "Generic (PLEG): container finished" podID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerID="ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a" exitCode=0
Sep 16 14:07:05.816275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816280    2580 generic.go:356] "Generic (PLEG): container finished" podID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerID="e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f" exitCode=0
Sep 16 14:07:05.816275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816285    2580 generic.go:356] "Generic (PLEG): container finished" podID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerID="6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175" exitCode=0
Sep 16 14:07:05.816562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f"}
Sep 16 14:07:05.816562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2"}
Sep 16 14:07:05.816562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a"}
Sep 16 14:07:05.816562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f"}
Sep 16 14:07:05.816562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:05.816455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175"}
Sep 16 14:07:06.116556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.116531    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.166702 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166665    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-web-config\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.166702 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166705    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-out\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.166929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-tls-assets\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.166929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166779    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm44n\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-kube-api-access-mm44n\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.166929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-cluster-tls-config\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.166929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166876    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.166929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.167188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166944    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-main-tls\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.167188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.166973    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-main-db\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.167188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.167013    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-volume\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.167188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.167039    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.167188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.167068    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.167188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.167107    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-metrics-client-ca\") pod \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\" (UID: \"e84dbbfb-5f6a-4528-98ac-72410772b4cf\") "
Sep 16 14:07:06.167751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.167713    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:07:06.167866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.167747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:06.168536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.168504    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:06.170207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.170172    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-out" (OuterVolumeSpecName: "config-out") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:07:06.170609 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.170574    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-kube-api-access-mm44n" (OuterVolumeSpecName: "kube-api-access-mm44n") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "kube-api-access-mm44n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:07:06.170835 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.170807    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-volume" (OuterVolumeSpecName: "config-volume") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:06.171340 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.171306    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:06.171892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.171860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:06.172177 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.172120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:06.172903 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.172883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:07:06.173526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.173497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:06.176768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.176725    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:06.182513 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.182490    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-web-config" (OuterVolumeSpecName: "web-config") pod "e84dbbfb-5f6a-4528-98ac-72410772b4cf" (UID: "e84dbbfb-5f6a-4528-98ac-72410772b4cf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:06.268275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268239    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-cluster-tls-config\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268270    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268285    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268299    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-main-tls\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268310    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-alertmanager-main-db\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268321    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268332    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268344    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268358    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e84dbbfb-5f6a-4528-98ac-72410772b4cf-metrics-client-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268369    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e84dbbfb-5f6a-4528-98ac-72410772b4cf-web-config\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268379    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e84dbbfb-5f6a-4528-98ac-72410772b4cf-config-out\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268390    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-tls-assets\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.268497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.268401    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm44n\" (UniqueName: \"kubernetes.io/projected/e84dbbfb-5f6a-4528-98ac-72410772b4cf-kube-api-access-mm44n\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:06.823758 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.823728    2580 generic.go:356] "Generic (PLEG): container finished" podID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerID="3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c" exitCode=0
Sep 16 14:07:06.823922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.823809    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.823922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.823810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c"}
Sep 16 14:07:06.823922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.823883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e84dbbfb-5f6a-4528-98ac-72410772b4cf","Type":"ContainerDied","Data":"5a9802985e9a3488674e956bbb548a59da7e3111d21c1cd75c14139a36906883"}
Sep 16 14:07:06.823922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.823916    2580 scope.go:117] "RemoveContainer" containerID="db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f"
Sep 16 14:07:06.832523 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.832413    2580 scope.go:117] "RemoveContainer" containerID="41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2"
Sep 16 14:07:06.840247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.840226    2580 scope.go:117] "RemoveContainer" containerID="ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a"
Sep 16 14:07:06.846766 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.846741    2580 scope.go:117] "RemoveContainer" containerID="3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c"
Sep 16 14:07:06.850580 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.850557    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:07:06.854822 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.854798    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:07:06.855021 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.855003    2580 scope.go:117] "RemoveContainer" containerID="e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f"
Sep 16 14:07:06.862050 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.862029    2580 scope.go:117] "RemoveContainer" containerID="6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175"
Sep 16 14:07:06.868919 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.868904    2580 scope.go:117] "RemoveContainer" containerID="6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd"
Sep 16 14:07:06.875761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.875743    2580 scope.go:117] "RemoveContainer" containerID="db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f"
Sep 16 14:07:06.876043 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:06.876024    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f\": container with ID starting with db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f not found: ID does not exist" containerID="db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f"
Sep 16 14:07:06.876082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876052    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f"} err="failed to get container status \"db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f\": rpc error: code = NotFound desc = could not find container \"db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f\": container with ID starting with db46d5c9a51004a6709e1133afd9cddca71f36edaae314d790e53281b87d603f not found: ID does not exist"
Sep 16 14:07:06.876082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876071    2580 scope.go:117] "RemoveContainer" containerID="41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2"
Sep 16 14:07:06.876340 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:06.876323    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2\": container with ID starting with 41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2 not found: ID does not exist" containerID="41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2"
Sep 16 14:07:06.876387 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876345    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2"} err="failed to get container status \"41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2\": rpc error: code = NotFound desc = could not find container \"41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2\": container with ID starting with 41a9e422a48b6d440a789d1b1bf75764843b8becc160909e95d610623ea074b2 not found: ID does not exist"
Sep 16 14:07:06.876387 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876358    2580 scope.go:117] "RemoveContainer" containerID="ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a"
Sep 16 14:07:06.876610 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:06.876592    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a\": container with ID starting with ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a not found: ID does not exist" containerID="ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a"
Sep 16 14:07:06.876671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876618    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a"} err="failed to get container status \"ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a\": rpc error: code = NotFound desc = could not find container \"ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a\": container with ID starting with ff6f5db6613ca121f0404048d9f1b29fca487598203238822232627842b1578a not found: ID does not exist"
Sep 16 14:07:06.876671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876640    2580 scope.go:117] "RemoveContainer" containerID="3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c"
Sep 16 14:07:06.876867 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:06.876852    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c\": container with ID starting with 3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c not found: ID does not exist" containerID="3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c"
Sep 16 14:07:06.876944 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876871    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c"} err="failed to get container status \"3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c\": rpc error: code = NotFound desc = could not find container \"3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c\": container with ID starting with 3d22aff0075a71ab3fa111f71925af7c68825eee7831397eb0f2b73438760e4c not found: ID does not exist"
Sep 16 14:07:06.876944 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.876884    2580 scope.go:117] "RemoveContainer" containerID="e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f"
Sep 16 14:07:06.877091 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:06.877076    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f\": container with ID starting with e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f not found: ID does not exist" containerID="e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f"
Sep 16 14:07:06.877148 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.877094    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f"} err="failed to get container status \"e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f\": rpc error: code = NotFound desc = could not find container \"e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f\": container with ID starting with e3ab55e46a02368aad9252fc7cbfaec2a13a54a73f23a4af8ded0292a3242d9f not found: ID does not exist"
Sep 16 14:07:06.877148 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.877106    2580 scope.go:117] "RemoveContainer" containerID="6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175"
Sep 16 14:07:06.877315 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:06.877300    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175\": container with ID starting with 6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175 not found: ID does not exist" containerID="6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175"
Sep 16 14:07:06.877351 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.877317    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175"} err="failed to get container status \"6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175\": rpc error: code = NotFound desc = could not find container \"6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175\": container with ID starting with 6a17911d908cdfbab2fd270b716d9f45ddeb95e149907e6d81450bace661d175 not found: ID does not exist"
Sep 16 14:07:06.877351 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.877327    2580 scope.go:117] "RemoveContainer" containerID="6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd"
Sep 16 14:07:06.877525 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:06.877509    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd\": container with ID starting with 6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd not found: ID does not exist" containerID="6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd"
Sep 16 14:07:06.877576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.877531    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd"} err="failed to get container status \"6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd\": rpc error: code = NotFound desc = could not find container \"6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd\": container with ID starting with 6214bf28d298c66981f44371c0d961050eb7f59e50b431e95f3944651e1772cd not found: ID does not exist"
Sep 16 14:07:06.884398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884374    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:07:06.884664 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884653    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="prom-label-proxy"
Sep 16 14:07:06.884703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884666    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="prom-label-proxy"
Sep 16 14:07:06.884703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884676    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy"
Sep 16 14:07:06.884703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884681    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy"
Sep 16 14:07:06.884703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884689    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-metric"
Sep 16 14:07:06.884703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884694    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-metric"
Sep 16 14:07:06.884703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884704    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="config-reloader"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884708    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="config-reloader"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884721    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="alertmanager"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884726    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="alertmanager"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884732    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="init-config-reloader"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884739    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="init-config-reloader"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884748    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" containerName="console"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884753    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" containerName="console"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884760    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-web"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884765    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-web"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884804    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884811    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="prom-label-proxy"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884817    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4cdd4d0-e47c-4a76-8609-3a910f1b25e5" containerName="console"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884826    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-web"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884832    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="alertmanager"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884836    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="config-reloader"
Sep 16 14:07:06.884889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.884841    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" containerName="kube-rbac-proxy-metric"
Sep 16 14:07:06.890162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.889473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.893220 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.893195    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 14:07:06.893429 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.893412    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 14:07:06.893611 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.893596    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 14:07:06.893783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.893769    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 14:07:06.893996 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.893980    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 14:07:06.894096 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.894081    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-45nr6\""
Sep 16 14:07:06.894481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.894462    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 14:07:06.894906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.894881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 14:07:06.895172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.895156    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 14:07:06.899008 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.898986    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 14:07:06.904025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.904004    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:07:06.974596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhmlg\" (UniqueName: \"kubernetes.io/projected/725f126e-01b0-4981-8c71-8a3311ec2f97-kube-api-access-dhmlg\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/725f126e-01b0-4981-8c71-8a3311ec2f97-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974914 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974914 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/725f126e-01b0-4981-8c71-8a3311ec2f97-tls-assets\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974914 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/725f126e-01b0-4981-8c71-8a3311ec2f97-config-out\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.974914 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-config-volume\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.975114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.974984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/725f126e-01b0-4981-8c71-8a3311ec2f97-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.975114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.975020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-web-config\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.975114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.975053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:06.975114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:06.975077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/725f126e-01b0-4981-8c71-8a3311ec2f97-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076195 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076195 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhmlg\" (UniqueName: \"kubernetes.io/projected/725f126e-01b0-4981-8c71-8a3311ec2f97-kube-api-access-dhmlg\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/725f126e-01b0-4981-8c71-8a3311ec2f97-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076542 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076542 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/725f126e-01b0-4981-8c71-8a3311ec2f97-tls-assets\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076542 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/725f126e-01b0-4981-8c71-8a3311ec2f97-config-out\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-config-volume\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/725f126e-01b0-4981-8c71-8a3311ec2f97-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-web-config\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.076882 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.076664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/725f126e-01b0-4981-8c71-8a3311ec2f97-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.077347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.077019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/725f126e-01b0-4981-8c71-8a3311ec2f97-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.077470 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.077417    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/725f126e-01b0-4981-8c71-8a3311ec2f97-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.079347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.079304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/725f126e-01b0-4981-8c71-8a3311ec2f97-config-out\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.079449 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.079402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/725f126e-01b0-4981-8c71-8a3311ec2f97-tls-assets\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.079515 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.079498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.079593 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.079563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.079707 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.079618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.079796 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.079774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-config-volume\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.080020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.079986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/725f126e-01b0-4981-8c71-8a3311ec2f97-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.080103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.080000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.080208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.080188    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-web-config\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.081521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.081505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/725f126e-01b0-4981-8c71-8a3311ec2f97-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.085494 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.085475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhmlg\" (UniqueName: \"kubernetes.io/projected/725f126e-01b0-4981-8c71-8a3311ec2f97-kube-api-access-dhmlg\") pod \"alertmanager-main-0\" (UID: \"725f126e-01b0-4981-8c71-8a3311ec2f97\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.201640 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.201607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:07:07.478486 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.478459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:07:07.480942 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:07:07.480917    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725f126e_01b0_4981_8c71_8a3311ec2f97.slice/crio-efe0a70ce4ac4c0a2b08879b9020351f09d1fd753ef485d279a61de654228dda WatchSource:0}: Error finding container efe0a70ce4ac4c0a2b08879b9020351f09d1fd753ef485d279a61de654228dda: Status 404 returned error can't find the container with id efe0a70ce4ac4c0a2b08879b9020351f09d1fd753ef485d279a61de654228dda
Sep 16 14:07:07.829356 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.829271    2580 generic.go:356] "Generic (PLEG): container finished" podID="725f126e-01b0-4981-8c71-8a3311ec2f97" containerID="bbcfd6e861915f9f806a82f983a3f1f95cf0e9ebfb878bda75f6177d9c8ea200" exitCode=0
Sep 16 14:07:07.829356 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.829325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerDied","Data":"bbcfd6e861915f9f806a82f983a3f1f95cf0e9ebfb878bda75f6177d9c8ea200"}
Sep 16 14:07:07.829521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:07.829390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerStarted","Data":"efe0a70ce4ac4c0a2b08879b9020351f09d1fd753ef485d279a61de654228dda"}
Sep 16 14:07:08.005254 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.005227    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e84dbbfb-5f6a-4528-98ac-72410772b4cf" path="/var/lib/kubelet/pods/e84dbbfb-5f6a-4528-98ac-72410772b4cf/volumes"
Sep 16 14:07:08.837830 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.837789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerStarted","Data":"b75d8e07a83d790fe41d56853ecfedf5ef76c4a94b26476f68daaf4e24c93e68"}
Sep 16 14:07:08.838272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.837889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerStarted","Data":"5b8adda1130dd8816412fcf410c71b96ebcbdef6b611d0914ad785dcfdf99872"}
Sep 16 14:07:08.838272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.837922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerStarted","Data":"fdc5c5968f99c533840a399176366bec78769c2c6930e0d3ee14872815722254"}
Sep 16 14:07:08.838272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.837956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerStarted","Data":"7030210cc85602d09cd10ce75dee98012b33c92c5c01178575b3297a87e5ef3a"}
Sep 16 14:07:08.838272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.838001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerStarted","Data":"7868006f3ce8edafe9110a9c4bdb05735b1ce73c2b99ebe078a11cc668bc09db"}
Sep 16 14:07:08.838272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.838028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"725f126e-01b0-4981-8c71-8a3311ec2f97","Type":"ContainerStarted","Data":"d9cb3e3784505821478f73df30458619fe02b1d1be9f3711e77b6df117508747"}
Sep 16 14:07:08.873304 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:08.873232    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.858891992 podStartE2EDuration="2.873210018s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.318026ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:07:08.870821559 +0000 UTC m=+159.358831635" watchObservedRunningTime="2026-09-16 14:07:08.873210018 +0000 UTC m=+159.361220095"
Sep 16 14:07:09.231329 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.231295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:07:09.231662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.231636    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="prometheus" containerID="cri-o://390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4" gracePeriod=600
Sep 16 14:07:09.231743 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.231658    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy" containerID="cri-o://48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b" gracePeriod=600
Sep 16 14:07:09.231834 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.231813    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-web" containerID="cri-o://b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19" gracePeriod=600
Sep 16 14:07:09.231834 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.231824    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963" gracePeriod=600
Sep 16 14:07:09.231941 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.231902    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="thanos-sidecar" containerID="cri-o://7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14" gracePeriod=600
Sep 16 14:07:09.232000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.231942    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="config-reloader" containerID="cri-o://fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac" gracePeriod=600
Sep 16 14:07:09.486168 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.480868    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:09.596822 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.596790    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-metrics-client-certs\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.596968 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.596900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-web-config\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.596968 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.596932    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-thanos-prometheus-http-client-file\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.596968 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.596956    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjzdh\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-kube-api-access-rjzdh\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597071    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-tls\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597120    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-kubelet-serving-ca-bundle\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597216 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597166    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597216 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597193    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-2\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597479    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:09.597585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-1\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597580    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-kube-rbac-proxy\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-grpc-tls\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597660    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config-out\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-trusted-ca-bundle\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597734    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-tls-assets\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597760    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-0\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.597794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-metrics-client-ca\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.598163 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-serving-certs-ca-bundle\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.598163 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597880    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-db\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.598163 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597919    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\" (UID: \"5c5207a6-8974-4dec-82bb-c461a7bd7b2c\") "
Sep 16 14:07:09.598319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.598267    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.598372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597791    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:09.598429 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.597830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:09.598429 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.598342    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:09.598918 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.598854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:09.600187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.600161    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:09.600641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.600510    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.600641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.600525    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:07:09.600641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.600616    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.600928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.600894    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-kube-api-access-rjzdh" (OuterVolumeSpecName: "kube-api-access-rjzdh") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "kube-api-access-rjzdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:07:09.601214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.601196    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:07:09.601909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.601883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.602000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.601905    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.602000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.601917    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.602000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.601966    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.602187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.601992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:07:09.602246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.602197    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config-out" (OuterVolumeSpecName: "config-out") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:07:09.602292 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.602240    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.602323 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.602289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config" (OuterVolumeSpecName: "config") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.610380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.610359    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-web-config" (OuterVolumeSpecName: "web-config") pod "5c5207a6-8974-4dec-82bb-c461a7bd7b2c" (UID: "5c5207a6-8974-4dec-82bb-c461a7bd7b2c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:07:09.698886 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698850    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-kube-rbac-proxy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.698886 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698885    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-grpc-tls\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698899    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698910    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-config-out\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698921    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698934    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-tls-assets\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698946    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698959    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-metrics-client-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698973    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.698987    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-db\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699000    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699013    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-metrics-client-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699026    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-web-config\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699039    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699051    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjzdh\" (UniqueName: \"kubernetes.io/projected/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-kube-api-access-rjzdh\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699064    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699077    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699091    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.699587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.699104    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c5207a6-8974-4dec-82bb-c461a7bd7b2c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:07:09.845176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845107    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963" exitCode=0
Sep 16 14:07:09.845176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845125    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b" exitCode=0
Sep 16 14:07:09.845176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845131    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19" exitCode=0
Sep 16 14:07:09.845176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845149    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14" exitCode=0
Sep 16 14:07:09.845176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845154    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac" exitCode=0
Sep 16 14:07:09.845176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845159    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4" exitCode=0
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"}
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845238    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"}
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"}
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"}
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845355    2580 scope.go:117] "RemoveContainer" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"}
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"}
Sep 16 14:07:09.845767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.845417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c5207a6-8974-4dec-82bb-c461a7bd7b2c","Type":"ContainerDied","Data":"538d612a95956446b8998e2f79ea2ff45241e1bd5c0b21c5b493b77b55790a88"}
Sep 16 14:07:09.853930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.853779    2580 scope.go:117] "RemoveContainer" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.862884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.862864    2580 scope.go:117] "RemoveContainer" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.869367 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.869348    2580 scope.go:117] "RemoveContainer" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.871811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.871787    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:07:09.876831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.876810    2580 scope.go:117] "RemoveContainer" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.876953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.876938    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:07:09.883111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.883094    2580 scope.go:117] "RemoveContainer" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.889926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.889901    2580 scope.go:117] "RemoveContainer" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.896470 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.896455    2580 scope.go:117] "RemoveContainer" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.896745 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:09.896731    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": container with ID starting with a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963 not found: ID does not exist" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.896792 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.896751    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"} err="failed to get container status \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": rpc error: code = NotFound desc = could not find container \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": container with ID starting with a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963 not found: ID does not exist"
Sep 16 14:07:09.896792 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.896766    2580 scope.go:117] "RemoveContainer" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.896993 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:09.896978    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": container with ID starting with 48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b not found: ID does not exist" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.897046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897000    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"} err="failed to get container status \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": rpc error: code = NotFound desc = could not find container \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": container with ID starting with 48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b not found: ID does not exist"
Sep 16 14:07:09.897046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897018    2580 scope.go:117] "RemoveContainer" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.897272 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:09.897255    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": container with ID starting with b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19 not found: ID does not exist" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.897341 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897278    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"} err="failed to get container status \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": rpc error: code = NotFound desc = could not find container \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": container with ID starting with b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19 not found: ID does not exist"
Sep 16 14:07:09.897341 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897294    2580 scope.go:117] "RemoveContainer" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.897529 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:09.897514    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": container with ID starting with 7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14 not found: ID does not exist" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.897573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897531    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"} err="failed to get container status \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": rpc error: code = NotFound desc = could not find container \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": container with ID starting with 7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14 not found: ID does not exist"
Sep 16 14:07:09.897573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897544    2580 scope.go:117] "RemoveContainer" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.897784 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:09.897765    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": container with ID starting with fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac not found: ID does not exist" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.897846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897792    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"} err="failed to get container status \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": rpc error: code = NotFound desc = could not find container \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": container with ID starting with fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac not found: ID does not exist"
Sep 16 14:07:09.897846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.897815    2580 scope.go:117] "RemoveContainer" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.898030 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:09.898013    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": container with ID starting with 390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4 not found: ID does not exist" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.898081 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898034    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"} err="failed to get container status \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": rpc error: code = NotFound desc = could not find container \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": container with ID starting with 390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4 not found: ID does not exist"
Sep 16 14:07:09.898081 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898047    2580 scope.go:117] "RemoveContainer" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.898311 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:07:09.898294    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": container with ID starting with 44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87 not found: ID does not exist" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.898359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898315    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"} err="failed to get container status \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": rpc error: code = NotFound desc = could not find container \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": container with ID starting with 44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87 not found: ID does not exist"
Sep 16 14:07:09.898359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898330    2580 scope.go:117] "RemoveContainer" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.898578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898560    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"} err="failed to get container status \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": rpc error: code = NotFound desc = could not find container \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": container with ID starting with a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963 not found: ID does not exist"
Sep 16 14:07:09.898620 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898579    2580 scope.go:117] "RemoveContainer" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.898771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898754    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"} err="failed to get container status \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": rpc error: code = NotFound desc = could not find container \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": container with ID starting with 48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b not found: ID does not exist"
Sep 16 14:07:09.898815 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898771    2580 scope.go:117] "RemoveContainer" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.898968 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898951    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"} err="failed to get container status \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": rpc error: code = NotFound desc = could not find container \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": container with ID starting with b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19 not found: ID does not exist"
Sep 16 14:07:09.899009 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.898967    2580 scope.go:117] "RemoveContainer" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.899170 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899149    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"} err="failed to get container status \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": rpc error: code = NotFound desc = could not find container \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": container with ID starting with 7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14 not found: ID does not exist"
Sep 16 14:07:09.899170 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899164    2580 scope.go:117] "RemoveContainer" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.899385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899371    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"} err="failed to get container status \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": rpc error: code = NotFound desc = could not find container \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": container with ID starting with fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac not found: ID does not exist"
Sep 16 14:07:09.899385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899384    2580 scope.go:117] "RemoveContainer" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.899607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899591    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"} err="failed to get container status \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": rpc error: code = NotFound desc = could not find container \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": container with ID starting with 390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4 not found: ID does not exist"
Sep 16 14:07:09.899653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899607    2580 scope.go:117] "RemoveContainer" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.899829 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899806    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"} err="failed to get container status \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": rpc error: code = NotFound desc = could not find container \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": container with ID starting with 44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87 not found: ID does not exist"
Sep 16 14:07:09.899923 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.899831    2580 scope.go:117] "RemoveContainer" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.900189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.900133    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"} err="failed to get container status \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": rpc error: code = NotFound desc = could not find container \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": container with ID starting with a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963 not found: ID does not exist"
Sep 16 14:07:09.900299 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.900193    2580 scope.go:117] "RemoveContainer" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.900511 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.900476    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"} err="failed to get container status \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": rpc error: code = NotFound desc = could not find container \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": container with ID starting with 48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b not found: ID does not exist"
Sep 16 14:07:09.900511 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.900510    2580 scope.go:117] "RemoveContainer" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.901095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901062    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"} err="failed to get container status \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": rpc error: code = NotFound desc = could not find container \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": container with ID starting with b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19 not found: ID does not exist"
Sep 16 14:07:09.901095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901089    2580 scope.go:117] "RemoveContainer" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.901447 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901423    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"} err="failed to get container status \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": rpc error: code = NotFound desc = could not find container \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": container with ID starting with 7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14 not found: ID does not exist"
Sep 16 14:07:09.901447 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901447    2580 scope.go:117] "RemoveContainer" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.901691 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901675    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"} err="failed to get container status \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": rpc error: code = NotFound desc = could not find container \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": container with ID starting with fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac not found: ID does not exist"
Sep 16 14:07:09.901691 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901690    2580 scope.go:117] "RemoveContainer" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.901937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901919    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:07:09.902015 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.901915    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"} err="failed to get container status \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": rpc error: code = NotFound desc = could not find container \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": container with ID starting with 390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4 not found: ID does not exist"
Sep 16 14:07:09.902077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902015    2580 scope.go:117] "RemoveContainer" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.902268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902249    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"} err="failed to get container status \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": rpc error: code = NotFound desc = could not find container \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": container with ID starting with 44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87 not found: ID does not exist"
Sep 16 14:07:09.902268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902267    2580 scope.go:117] "RemoveContainer" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.902377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902303    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="config-reloader"
Sep 16 14:07:09.902377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902320    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="config-reloader"
Sep 16 14:07:09.902377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902337    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="prometheus"
Sep 16 14:07:09.902377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902346    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="prometheus"
Sep 16 14:07:09.902377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902361    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="init-config-reloader"
Sep 16 14:07:09.902377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902369    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="init-config-reloader"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902387    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="thanos-sidecar"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902395    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="thanos-sidecar"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902405    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-web"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902414    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-web"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902427    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902434    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902443    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-thanos"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902453    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-thanos"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902458    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"} err="failed to get container status \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": rpc error: code = NotFound desc = could not find container \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": container with ID starting with a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963 not found: ID does not exist"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902479    2580 scope.go:117] "RemoveContainer" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902524    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="prometheus"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902536    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-web"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902545    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="thanos-sidecar"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902553    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy-thanos"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902563    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="kube-rbac-proxy"
Sep 16 14:07:09.902654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902572    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" containerName="config-reloader"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902662    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"} err="failed to get container status \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": rpc error: code = NotFound desc = could not find container \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": container with ID starting with 48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b not found: ID does not exist"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902676    2580 scope.go:117] "RemoveContainer" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902863    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"} err="failed to get container status \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": rpc error: code = NotFound desc = could not find container \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": container with ID starting with b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19 not found: ID does not exist"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.902879    2580 scope.go:117] "RemoveContainer" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903057    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"} err="failed to get container status \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": rpc error: code = NotFound desc = could not find container \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": container with ID starting with 7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14 not found: ID does not exist"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903070    2580 scope.go:117] "RemoveContainer" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903328    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"} err="failed to get container status \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": rpc error: code = NotFound desc = could not find container \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": container with ID starting with fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac not found: ID does not exist"
Sep 16 14:07:09.903407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903343    2580 scope.go:117] "RemoveContainer" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.903782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903572    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"} err="failed to get container status \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": rpc error: code = NotFound desc = could not find container \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": container with ID starting with 390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4 not found: ID does not exist"
Sep 16 14:07:09.903782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903588    2580 scope.go:117] "RemoveContainer" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.903878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903785    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"} err="failed to get container status \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": rpc error: code = NotFound desc = could not find container \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": container with ID starting with 44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87 not found: ID does not exist"
Sep 16 14:07:09.903878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903797    2580 scope.go:117] "RemoveContainer" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.904015 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.903997    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"} err="failed to get container status \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": rpc error: code = NotFound desc = could not find container \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": container with ID starting with a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963 not found: ID does not exist"
Sep 16 14:07:09.904015 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904013    2580 scope.go:117] "RemoveContainer" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.904225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904208    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"} err="failed to get container status \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": rpc error: code = NotFound desc = could not find container \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": container with ID starting with 48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b not found: ID does not exist"
Sep 16 14:07:09.904225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904222    2580 scope.go:117] "RemoveContainer" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.904421 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904407    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"} err="failed to get container status \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": rpc error: code = NotFound desc = could not find container \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": container with ID starting with b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19 not found: ID does not exist"
Sep 16 14:07:09.904421 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904420    2580 scope.go:117] "RemoveContainer" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.904623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904608    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"} err="failed to get container status \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": rpc error: code = NotFound desc = could not find container \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": container with ID starting with 7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14 not found: ID does not exist"
Sep 16 14:07:09.904656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904623    2580 scope.go:117] "RemoveContainer" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.904830 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904816    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"} err="failed to get container status \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": rpc error: code = NotFound desc = could not find container \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": container with ID starting with fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac not found: ID does not exist"
Sep 16 14:07:09.904874 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.904829    2580 scope.go:117] "RemoveContainer" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.905040 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905019    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"} err="failed to get container status \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": rpc error: code = NotFound desc = could not find container \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": container with ID starting with 390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4 not found: ID does not exist"
Sep 16 14:07:09.905078 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905041    2580 scope.go:117] "RemoveContainer" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.905282 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905264    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"} err="failed to get container status \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": rpc error: code = NotFound desc = could not find container \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": container with ID starting with 44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87 not found: ID does not exist"
Sep 16 14:07:09.905359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905283    2580 scope.go:117] "RemoveContainer" containerID="a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"
Sep 16 14:07:09.905509 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905492    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963"} err="failed to get container status \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": rpc error: code = NotFound desc = could not find container \"a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963\": container with ID starting with a8736694c3c9786571d0b0a6fb8e505058b5a6a13f7d33d6e8f7a546cc255963 not found: ID does not exist"
Sep 16 14:07:09.905566 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905509    2580 scope.go:117] "RemoveContainer" containerID="48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"
Sep 16 14:07:09.905719 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905699    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b"} err="failed to get container status \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": rpc error: code = NotFound desc = could not find container \"48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b\": container with ID starting with 48e5c48237601c5e7624947191c939c8ab7338ef0d896890b7027bf711cf7f8b not found: ID does not exist"
Sep 16 14:07:09.905719 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905717    2580 scope.go:117] "RemoveContainer" containerID="b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"
Sep 16 14:07:09.905997 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.905936    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19"} err="failed to get container status \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": rpc error: code = NotFound desc = could not find container \"b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19\": container with ID starting with b37125d4e6bd8626edf9cbd8eb59873a9375ea5877d023395500b36c15484a19 not found: ID does not exist"
Sep 16 14:07:09.906073 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906001    2580 scope.go:117] "RemoveContainer" containerID="7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"
Sep 16 14:07:09.906304 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906286    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14"} err="failed to get container status \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": rpc error: code = NotFound desc = could not find container \"7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14\": container with ID starting with 7933e4c9205d654681d56db9ebc434f38483a11ec56e95ac386894b9bca62c14 not found: ID does not exist"
Sep 16 14:07:09.906304 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906304    2580 scope.go:117] "RemoveContainer" containerID="fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"
Sep 16 14:07:09.906556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906535    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac"} err="failed to get container status \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": rpc error: code = NotFound desc = could not find container \"fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac\": container with ID starting with fc11eaba8f9b1558cf4d4cda534a76b9857e714fedadf338ce8596c088a34eac not found: ID does not exist"
Sep 16 14:07:09.906556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906555    2580 scope.go:117] "RemoveContainer" containerID="390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"
Sep 16 14:07:09.906696 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:09.906786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906765    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4"} err="failed to get container status \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": rpc error: code = NotFound desc = could not find container \"390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4\": container with ID starting with 390b54a7980dc1a75373c66009e7118e93519fda2f4d63fece692aa821be5fe4 not found: ID does not exist"
Sep 16 14:07:09.906786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.906783    2580 scope.go:117] "RemoveContainer" containerID="44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"
Sep 16 14:07:09.907264 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.907110    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87"} err="failed to get container status \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": rpc error: code = NotFound desc = could not find container \"44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87\": container with ID starting with 44ff001cd34b2d33ca53492e2ef6f1251e4a090c89eb08f406d2e7d572b8bd87 not found: ID does not exist"
Sep 16 14:07:09.909457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.909436    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 14:07:09.909661 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.909645    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bknhso0dr3ehn\""
Sep 16 14:07:09.909661 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.909660    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 14:07:09.909785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.909737    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 14:07:09.909872 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.909840    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 14:07:09.909942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.909881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 14:07:09.910317 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.910300    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 14:07:09.910814 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.910798    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 14:07:09.911177 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.911159    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 14:07:09.911274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.911176    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 14:07:09.911648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.911629    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rf4np\""
Sep 16 14:07:09.911761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.911728    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 14:07:09.911834 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.911767    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 14:07:09.911926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.911911    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 14:07:09.914587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.914569    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 14:07:09.915674 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.915657    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 14:07:09.923780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:09.923761    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:07:10.001751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.001860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.001860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.001860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.001860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.001981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-config\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnbtd\" (UniqueName: \"kubernetes.io/projected/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-kube-api-access-vnbtd\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.002455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.002326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.006033 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.006013    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c5207a6-8974-4dec-82bb-c461a7bd7b2c" path="/var/lib/kubelet/pods/5c5207a6-8974-4dec-82bb-c461a7bd7b2c/volumes"
Sep 16 14:07:10.103009 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.102961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103009 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.102991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-config\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnbtd\" (UniqueName: \"kubernetes.io/projected/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-kube-api-access-vnbtd\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.103930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.103830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.104270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.104241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.104487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.104464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.105625 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.104948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.107056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.106661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.107056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.106798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.107056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.106970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.107406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.107366    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.107490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.107448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.107723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.107684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108169 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-config\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108800 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.108933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.108905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.111850 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.111833    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnbtd\" (UniqueName: \"kubernetes.io/projected/3a1c6f5b-9412-4aad-9442-3585f02aaf0d-kube-api-access-vnbtd\") pod \"prometheus-k8s-0\" (UID: \"3a1c6f5b-9412-4aad-9442-3585f02aaf0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.218245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.218220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:10.503027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.502999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:07:10.504935 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:07:10.504904    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1c6f5b_9412_4aad_9442_3585f02aaf0d.slice/crio-e1abcd032eadba311a7e995098499638f9e7810c6b746e8c56d149219afdd72e WatchSource:0}: Error finding container e1abcd032eadba311a7e995098499638f9e7810c6b746e8c56d149219afdd72e: Status 404 returned error can't find the container with id e1abcd032eadba311a7e995098499638f9e7810c6b746e8c56d149219afdd72e
Sep 16 14:07:10.850913 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.850841    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a1c6f5b-9412-4aad-9442-3585f02aaf0d" containerID="116961715be39e934c0b4849d0c70ac8f3fc6293ce93b6bee28b588c70f4a7ed" exitCode=0
Sep 16 14:07:10.851268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.850928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerDied","Data":"116961715be39e934c0b4849d0c70ac8f3fc6293ce93b6bee28b588c70f4a7ed"}
Sep 16 14:07:10.851268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:10.851007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerStarted","Data":"e1abcd032eadba311a7e995098499638f9e7810c6b746e8c56d149219afdd72e"}
Sep 16 14:07:11.859792 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:11.859754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerStarted","Data":"9702994ba71ac08453a7e4f2081fb8530adbc6e7c5fe23a53d927adb135cd3e2"}
Sep 16 14:07:11.860236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:11.859841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerStarted","Data":"621610eb9cd87364a21ef1a0103e0ad107a4b5997b12670232d8f4b0d6a14051"}
Sep 16 14:07:11.860236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:11.859889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerStarted","Data":"f944fa38593d88e861c0d068b51bb98b9e646bf70ed321dbfaa795f2ce94001c"}
Sep 16 14:07:11.860236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:11.859934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerStarted","Data":"b7705892b4b44ecd0d3e4687fcb53a88ed86cf72858c061e091dc3fd18f9c47a"}
Sep 16 14:07:11.860236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:11.859973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerStarted","Data":"6a897fa2a29c658bcee4074edb8ed397472149a62515316b3ad21fb0c06995f6"}
Sep 16 14:07:11.860236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:11.860012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a1c6f5b-9412-4aad-9442-3585f02aaf0d","Type":"ContainerStarted","Data":"10c97884252c4d3c7ad3f8cbc8ff0519b53ed9047c4a157e9a1d0efccc4d9f09"}
Sep 16 14:07:11.893453 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:11.893398    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.87161787 podStartE2EDuration="2.893380964s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.763094ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:07:11.890652642 +0000 UTC m=+162.378662717" watchObservedRunningTime="2026-09-16 14:07:11.893380964 +0000 UTC m=+162.381391039"
Sep 16 14:07:15.219065 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:15.219034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:07:16.335507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:16.335476    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:16.435419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:16.435385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:27.176501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:27.176413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:27.362525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:27.362479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:27.426236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:27.426198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:58.676323 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:58.676287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:58.752928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:58.752897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:07:59.560954 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:07:59.560921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:10.218797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:10.218760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:08:10.233617 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:10.233594    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:08:11.117674 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:11.117647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:08:19.138522 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:19.138475    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:19.256466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:19.256428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:29.687349 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:29.687296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:29.804984 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:29.804953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:31.044904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:31.044862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:31.162088 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:31.162054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:39.213595 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:39.213551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:39.315400 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:39.315364    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:40.500863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:40.500830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:40.601426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:40.601388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:08:40.686539 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:08:40.686516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:10.804245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:10.804212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:10.933011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:10.932981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:29.870884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:29.870857    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 14:09:43.467947 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:43.467909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:46.485615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:46.485572    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:47.864158 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:47.864091    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:48.015219 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:48.015187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:50.038248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:50.038199    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:50.178845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:50.178815    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:50.313756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:50.313689    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:50.458493 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:50.458456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:52.691005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:52.690963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:52.859520 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:52.859481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:54.303796 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:54.303737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:54.402466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:54.402419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:59.170674 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:59.170639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:09:59.227426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:09:59.227386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:00.424321 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:00.424287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:04.311701 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:04.311665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:04.411626 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:04.411592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:16.274990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:16.274963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:16.377043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:16.376994    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:22.353795 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:22.353754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:22.494507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:22.494436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:24.168606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:24.168573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:34.597549 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.597518    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"]
Sep 16 14:10:34.600957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.600938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.606464 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.606440    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 14:10:34.607620 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.607560    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 14:10:34.607685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.607620    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 14:10:34.607732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.607700    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:10:34.607813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.607797    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vbs78\""
Sep 16 14:10:34.614223 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.614207    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 14:10:34.626776 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.626754    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"]
Sep 16 14:10:34.695697 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.695675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/728deb49-1351-4c2c-bb93-aa0084090a73-metrics-cert\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.695793 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.695713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/728deb49-1351-4c2c-bb93-aa0084090a73-manager-config\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.695793 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.695740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqk8s\" (UniqueName: \"kubernetes.io/projected/728deb49-1351-4c2c-bb93-aa0084090a73-kube-api-access-dqk8s\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.695901 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.695837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/728deb49-1351-4c2c-bb93-aa0084090a73-cert\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.796901 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.796873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/728deb49-1351-4c2c-bb93-aa0084090a73-cert\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.797003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.796937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/728deb49-1351-4c2c-bb93-aa0084090a73-metrics-cert\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.797003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.796981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/728deb49-1351-4c2c-bb93-aa0084090a73-manager-config\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.797112 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.797018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqk8s\" (UniqueName: \"kubernetes.io/projected/728deb49-1351-4c2c-bb93-aa0084090a73-kube-api-access-dqk8s\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.797565 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.797545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/728deb49-1351-4c2c-bb93-aa0084090a73-manager-config\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.799283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.799262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/728deb49-1351-4c2c-bb93-aa0084090a73-cert\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.799436 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.799416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/728deb49-1351-4c2c-bb93-aa0084090a73-metrics-cert\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.814316 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.814290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqk8s\" (UniqueName: \"kubernetes.io/projected/728deb49-1351-4c2c-bb93-aa0084090a73-kube-api-access-dqk8s\") pod \"lws-controller-manager-654b6f4fd4-vd82q\" (UID: \"728deb49-1351-4c2c-bb93-aa0084090a73\") " pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:34.910806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:34.910758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:35.050485 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.050460    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9b67fddbb-khhkb"]
Sep 16 14:10:35.053882 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.053867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.057111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.057092    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 14:10:35.057236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.057213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 14:10:35.057236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.057222    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 14:10:35.058040 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.058024    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w6sw4\""
Sep 16 14:10:35.060037 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.060021    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 14:10:35.060538 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.060337    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 14:10:35.065287 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.065261    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 14:10:35.069416 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.069398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9b67fddbb-khhkb"]
Sep 16 14:10:35.199781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.199714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-oauth-serving-cert\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.199781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.199759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-serving-cert\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.199912 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.199842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-service-ca\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.199912 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.199884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-config\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.199912 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.199904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-trusted-ca-bundle\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.200013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.199928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lns\" (UniqueName: \"kubernetes.io/projected/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-kube-api-access-h2lns\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.200013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.199959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-oauth-config\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.232344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.231667    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"]
Sep 16 14:10:35.236704 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.236689    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:10:35.300603 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.300584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-service-ca\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.300731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.300623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-config\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.300731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.300639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-trusted-ca-bundle\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.300731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.300661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lns\" (UniqueName: \"kubernetes.io/projected/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-kube-api-access-h2lns\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.300731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.300681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-oauth-config\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.300907 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.300797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-oauth-serving-cert\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.300907 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.300877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-serving-cert\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.301392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.301364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-service-ca\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.301392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.301382    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-config\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.301546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.301526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-oauth-serving-cert\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.301622 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.301605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-trusted-ca-bundle\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.302863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.302840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-oauth-config\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.303038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.303022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-console-serving-cert\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.315768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.315747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lns\" (UniqueName: \"kubernetes.io/projected/f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b-kube-api-access-h2lns\") pod \"console-9b67fddbb-khhkb\" (UID: \"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b\") " pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.362950 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.362926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:35.364293 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.364266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:35.470627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.470584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:35.645027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.645002    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9b67fddbb-khhkb"]
Sep 16 14:10:35.648032 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:10:35.648008    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c89cb1_ee03_4ebb_b1d3_bd7bfa2c835b.slice/crio-b1d1d4f56e8922b580de88f5090b1141b95e6a63f871e7773ed8daf715d51302 WatchSource:0}: Error finding container b1d1d4f56e8922b580de88f5090b1141b95e6a63f871e7773ed8daf715d51302: Status 404 returned error can't find the container with id b1d1d4f56e8922b580de88f5090b1141b95e6a63f871e7773ed8daf715d51302
Sep 16 14:10:35.680416 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.680389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q" event={"ID":"728deb49-1351-4c2c-bb93-aa0084090a73","Type":"ContainerStarted","Data":"97fa8341d7b5ce96c953e7af28842ed92bb5cec3aeab4e71ec5cb4da5ca39d95"}
Sep 16 14:10:35.681672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:35.681520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b67fddbb-khhkb" event={"ID":"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b","Type":"ContainerStarted","Data":"b1d1d4f56e8922b580de88f5090b1141b95e6a63f871e7773ed8daf715d51302"}
Sep 16 14:10:36.340756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:36.340720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:36.441030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:36.440917    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:36.569268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:36.569232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:36.689599 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:36.689523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b67fddbb-khhkb" event={"ID":"f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b","Type":"ContainerStarted","Data":"7b27e40049c1619dcbc69d6f4963895487881be457ff7b0bc6b98aad176a9b6d"}
Sep 16 14:10:36.711957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:36.711913    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9b67fddbb-khhkb" podStartSLOduration=1.711897494 podStartE2EDuration="1.711897494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:36.710518791 +0000 UTC m=+367.198528867" watchObservedRunningTime="2026-09-16 14:10:36.711897494 +0000 UTC m=+367.199907569"
Sep 16 14:10:36.818068 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:36.818029    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:37.015378 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.015349    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"]
Sep 16 14:10:37.024326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.024305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.028733 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.028710    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kkfv7\""
Sep 16 14:10:37.028833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.028757    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 14:10:37.028833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.028815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 14:10:37.028959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.028833    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 14:10:37.028959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.028890    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 14:10:37.039298 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.039276    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"]
Sep 16 14:10:37.118517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.118490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3e88296-ead3-46fe-88d2-0f6f9a38b640-tmp\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.118665 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.118559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65z2w\" (UniqueName: \"kubernetes.io/projected/c3e88296-ead3-46fe-88d2-0f6f9a38b640-kube-api-access-65z2w\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.119473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.118858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3e88296-ead3-46fe-88d2-0f6f9a38b640-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.119473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.118938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3e88296-ead3-46fe-88d2-0f6f9a38b640-manifests\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.119473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.119074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3e88296-ead3-46fe-88d2-0f6f9a38b640-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.219911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.219877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3e88296-ead3-46fe-88d2-0f6f9a38b640-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.220106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.219927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3e88296-ead3-46fe-88d2-0f6f9a38b640-manifests\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.220106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.219964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3e88296-ead3-46fe-88d2-0f6f9a38b640-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.220106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.219994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3e88296-ead3-46fe-88d2-0f6f9a38b640-tmp\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.220106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.220041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65z2w\" (UniqueName: \"kubernetes.io/projected/c3e88296-ead3-46fe-88d2-0f6f9a38b640-kube-api-access-65z2w\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.220412 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.220389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3e88296-ead3-46fe-88d2-0f6f9a38b640-manifests\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.222694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.222668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3e88296-ead3-46fe-88d2-0f6f9a38b640-tmp\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.222821 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.222797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3e88296-ead3-46fe-88d2-0f6f9a38b640-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.222971 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.222948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3e88296-ead3-46fe-88d2-0f6f9a38b640-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.229225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.229206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65z2w\" (UniqueName: \"kubernetes.io/projected/c3e88296-ead3-46fe-88d2-0f6f9a38b640-kube-api-access-65z2w\") pod \"opendatahub-operator-controller-manager-77f6d9877-vt696\" (UID: \"c3e88296-ead3-46fe-88d2-0f6f9a38b640\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.229880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.229838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:37.330605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.330509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:37.336516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.336484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:37.491218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.491181    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:37.606680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.606549    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:37.703305 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.703265    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:37.835860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.835832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:37.913226 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:37.913202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"]
Sep 16 14:10:37.915697 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:10:37.915675    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e88296_ead3_46fe_88d2_0f6f9a38b640.slice/crio-fb875b1a5b72d821dba4a44a8372528566ddc1c6e9aef4b0d078d0fec4a2c1c9 WatchSource:0}: Error finding container fb875b1a5b72d821dba4a44a8372528566ddc1c6e9aef4b0d078d0fec4a2c1c9: Status 404 returned error can't find the container with id fb875b1a5b72d821dba4a44a8372528566ddc1c6e9aef4b0d078d0fec4a2c1c9
Sep 16 14:10:38.422547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:38.422505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:38.703526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:38.703438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q" event={"ID":"728deb49-1351-4c2c-bb93-aa0084090a73","Type":"ContainerStarted","Data":"eaccc3f16977953c1b735dcda8bc88c3d633ae2a5bcff174b280b759b41f5182"}
Sep 16 14:10:38.703942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:38.703539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:38.705017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:38.704983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696" event={"ID":"c3e88296-ead3-46fe-88d2-0f6f9a38b640","Type":"ContainerStarted","Data":"fb875b1a5b72d821dba4a44a8372528566ddc1c6e9aef4b0d078d0fec4a2c1c9"}
Sep 16 14:10:38.741046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:38.740765    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q" podStartSLOduration=2.269454684 podStartE2EDuration="4.740732825s" totalImagesPullingTime="2.471278141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:38.739425001 +0000 UTC m=+369.227435079" watchObservedRunningTime="2026-09-16 14:10:38.740732825 +0000 UTC m=+369.228742902"
Sep 16 14:10:40.716580 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:40.716555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696" event={"ID":"c3e88296-ead3-46fe-88d2-0f6f9a38b640","Type":"ContainerStarted","Data":"4b792c22ee04af05800364717198ff17ddc03a223fd1748b57071879fdc749a8"}
Sep 16 14:10:41.722865 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:41.722837    2580 generic.go:356] "Generic (PLEG): container finished" podID="c3e88296-ead3-46fe-88d2-0f6f9a38b640" containerID="4b792c22ee04af05800364717198ff17ddc03a223fd1748b57071879fdc749a8" exitCode=0
Sep 16 14:10:41.723173 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:41.722927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696" event={"ID":"c3e88296-ead3-46fe-88d2-0f6f9a38b640","Type":"ContainerDied","Data":"4b792c22ee04af05800364717198ff17ddc03a223fd1748b57071879fdc749a8"}
Sep 16 14:10:42.730387 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:42.730352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696" event={"ID":"c3e88296-ead3-46fe-88d2-0f6f9a38b640","Type":"ContainerStarted","Data":"b1328b1ac798237ffc682c89fbd7fee92d8923c0931409fea8f641372f0322ca"}
Sep 16 14:10:42.730800 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:42.730497    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:42.772539 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:42.772490    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696" podStartSLOduration=2.395037417 podStartE2EDuration="6.772476105s" totalImagesPullingTime="3.383466605s" totalInitContainerRuntime="993.972083ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:42.769938712 +0000 UTC m=+373.257948787" watchObservedRunningTime="2026-09-16 14:10:42.772476105 +0000 UTC m=+373.260486180"
Sep 16 14:10:45.363584 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:45.363545    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:45.364008 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:45.363610    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:45.368541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:45.368514    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:45.747890 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:45.747855    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9b67fddbb-khhkb"
Sep 16 14:10:45.912369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:45.912337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:46.012982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:46.012911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:49.713012 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:49.712984    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-654b6f4fd4-vd82q"
Sep 16 14:10:53.738367 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:53.738327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-vt696"
Sep 16 14:10:57.930012 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:57.929978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:10:58.171482 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:10:58.171451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:01.329430 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.329403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:01.449491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.449461    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"]
Sep 16 14:11:01.457120 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.457099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.462013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.461996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 14:11:01.462768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.462679    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 14:11:01.462768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.462710    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rcfld\""
Sep 16 14:11:01.462768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.462723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 14:11:01.462768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.462757    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 14:11:01.484650 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.484629    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"]
Sep 16 14:11:01.508794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.508769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483a6573-d5b6-4526-9406-878ea7adeebb-tmp\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.508895 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.508844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7t8j\" (UniqueName: \"kubernetes.io/projected/483a6573-d5b6-4526-9406-878ea7adeebb-kube-api-access-b7t8j\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.508957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.508903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/483a6573-d5b6-4526-9406-878ea7adeebb-tls-certs\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.610087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.610037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483a6573-d5b6-4526-9406-878ea7adeebb-tmp\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.610208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.610092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7t8j\" (UniqueName: \"kubernetes.io/projected/483a6573-d5b6-4526-9406-878ea7adeebb-kube-api-access-b7t8j\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.610208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.610121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/483a6573-d5b6-4526-9406-878ea7adeebb-tls-certs\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.612294 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.612274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483a6573-d5b6-4526-9406-878ea7adeebb-tmp\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.612439 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.612421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/483a6573-d5b6-4526-9406-878ea7adeebb-tls-certs\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.624065 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.624040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7t8j\" (UniqueName: \"kubernetes.io/projected/483a6573-d5b6-4526-9406-878ea7adeebb-kube-api-access-b7t8j\") pod \"kube-auth-proxy-5b574975b8-5f7w7\" (UID: \"483a6573-d5b6-4526-9406-878ea7adeebb\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:01.766563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:01.766542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"
Sep 16 14:11:02.049972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:02.049952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7"]
Sep 16 14:11:02.052334 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:11:02.052309    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483a6573_d5b6_4526_9406_878ea7adeebb.slice/crio-e400424662b8d9e49cf4cbdb70b45bbb0df497ba55a73d8159ac6ce932ec4e96 WatchSource:0}: Error finding container e400424662b8d9e49cf4cbdb70b45bbb0df497ba55a73d8159ac6ce932ec4e96: Status 404 returned error can't find the container with id e400424662b8d9e49cf4cbdb70b45bbb0df497ba55a73d8159ac6ce932ec4e96
Sep 16 14:11:02.770311 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:02.770283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:02.821115 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:02.821088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7" event={"ID":"483a6573-d5b6-4526-9406-878ea7adeebb","Type":"ContainerStarted","Data":"e400424662b8d9e49cf4cbdb70b45bbb0df497ba55a73d8159ac6ce932ec4e96"}
Sep 16 14:11:03.304771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:03.304749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:03.387327 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:03.387303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:04.548700 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:04.548670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:06.841536 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:06.841514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7" event={"ID":"483a6573-d5b6-4526-9406-878ea7adeebb","Type":"ContainerStarted","Data":"1599b091984f1a6f896efb156e5851dfcaeebf05ed299bebd567ee5aeee8c3a0"}
Sep 16 14:11:06.864864 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:06.864819    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b574975b8-5f7w7" podStartSLOduration=1.136425802 podStartE2EDuration="5.864779268s" totalImagesPullingTime="4.728353466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:06.863020255 +0000 UTC m=+397.351030332" watchObservedRunningTime="2026-09-16 14:11:06.864779268 +0000 UTC m=+397.352789344"
Sep 16 14:11:07.995690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:07.995649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:08.946628 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:08.946592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:09.070426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:09.070389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:09.183409 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:09.183366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:09.285434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:09.285397    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:09.498585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:09.498546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:09.727526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:09.727495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:10.258358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:10.258322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:19.313234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:19.313201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:19.477470 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:19.477442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:20.031910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:20.031873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:20.708790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:20.708756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:11:20.819561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:11:20.819520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:09.933557 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:09.933520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:10.030750 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:10.030720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:22.404773 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:22.404736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:27.030768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:27.030728    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:27.876928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:27.876890    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:30.697680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:30.697649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:30.843318 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:30.843281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:30.944248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:30.944223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:31.425255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:31.425226    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:31.842000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:31.841965    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:33.138319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:33.138270    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:33.404666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:33.404573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:35.302775 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:35.302741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:35.397556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:35.397524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:37.447773 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:37.447733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:38.910501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:38.910464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:41.556427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:41.556392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:41.647587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:41.647559    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:42.056530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.056499    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:42.063795 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.063770    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"]
Sep 16 14:12:42.066809 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.066796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.069613 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.069589    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 14:12:42.069722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.069632    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 14:12:42.069722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.069644    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 14:12:42.070388 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.070372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 14:12:42.070388 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.070383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j72bn\""
Sep 16 14:12:42.075589 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.075568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"]
Sep 16 14:12:42.162887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.162866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxdlb\" (UniqueName: \"kubernetes.io/projected/57f4ef0e-c570-4775-b965-1013161e7387-kube-api-access-xxdlb\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.163002 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.162977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/57f4ef0e-c570-4775-b965-1013161e7387-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.163002 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.163000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/57f4ef0e-c570-4775-b965-1013161e7387-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.256209 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.256182    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:42.263698 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.263677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/57f4ef0e-c570-4775-b965-1013161e7387-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.263794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.263700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/57f4ef0e-c570-4775-b965-1013161e7387-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.263794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.263734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxdlb\" (UniqueName: \"kubernetes.io/projected/57f4ef0e-c570-4775-b965-1013161e7387-kube-api-access-xxdlb\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.264384 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.264368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/57f4ef0e-c570-4775-b965-1013161e7387-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.266039 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.266024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/57f4ef0e-c570-4775-b965-1013161e7387-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.272774 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.272751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxdlb\" (UniqueName: \"kubernetes.io/projected/57f4ef0e-c570-4775-b965-1013161e7387-kube-api-access-xxdlb\") pod \"kuadrant-console-plugin-fbdd4f445-9jsjk\" (UID: \"57f4ef0e-c570-4775-b965-1013161e7387\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.387378 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.387327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"
Sep 16 14:12:42.655942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:42.655886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk"]
Sep 16 14:12:42.658374 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:12:42.658341    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57f4ef0e_c570_4775_b965_1013161e7387.slice/crio-208f0365fffaabda6ccd658dc1b679b5ef59d016d4d1f5735ecc73c7c1b2bdbb WatchSource:0}: Error finding container 208f0365fffaabda6ccd658dc1b679b5ef59d016d4d1f5735ecc73c7c1b2bdbb: Status 404 returned error can't find the container with id 208f0365fffaabda6ccd658dc1b679b5ef59d016d4d1f5735ecc73c7c1b2bdbb
Sep 16 14:12:43.266288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:43.266255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk" event={"ID":"57f4ef0e-c570-4775-b965-1013161e7387","Type":"ContainerStarted","Data":"208f0365fffaabda6ccd658dc1b679b5ef59d016d4d1f5735ecc73c7c1b2bdbb"}
Sep 16 14:12:45.280932 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:45.280902    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:45.378465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:45.378440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:46.244617 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:46.244564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:46.344070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:46.344035    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:50.305966 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:50.305929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk" event={"ID":"57f4ef0e-c570-4775-b965-1013161e7387","Type":"ContainerStarted","Data":"02e584099b5eceae11b2e527bde375a188380a0e698a875d4baec3f9fb177aa2"}
Sep 16 14:12:50.327422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:50.327375    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9jsjk" podStartSLOduration=1.425082836 podStartE2EDuration="8.327358854s" totalImagesPullingTime="6.902276018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:50.325875999 +0000 UTC m=+500.813886074" watchObservedRunningTime="2026-09-16 14:12:50.327358854 +0000 UTC m=+500.815368928"
Sep 16 14:12:51.974469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:51.974400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:52.123610 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:52.123583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:56.711072 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:56.711039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:56.809252 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:56.809217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:59.486985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:59.486940    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:12:59.594911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:12:59.594873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:04.922325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:04.922288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:05.022175 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:05.022130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:05.488734 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:05.488700    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:05.589907 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:05.589877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:05.733396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:05.733369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:05.846480 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:05.846402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:11.224826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:11.224795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:11.320432 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:11.320404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:16.052444 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:16.052408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:16.186493 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:16.186461    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:22.996803 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:22.996724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:23.182716 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:23.182683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:25.963123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:25.963090    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:26.097614 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.097582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:26.292468 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.292404    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:13:26.294833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.294818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.298131 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.298110    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 14:13:26.377860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.377838    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:13:26.411705 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.411678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe9b18fc-ba30-48ce-b927-953c794ea102-config-file\") pod \"limitador-limitador-84994f7ffc-crfbz\" (UID: \"fe9b18fc-ba30-48ce-b927-953c794ea102\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.411813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.411708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxlww\" (UniqueName: \"kubernetes.io/projected/fe9b18fc-ba30-48ce-b927-953c794ea102-kube-api-access-bxlww\") pod \"limitador-limitador-84994f7ffc-crfbz\" (UID: \"fe9b18fc-ba30-48ce-b927-953c794ea102\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.431838 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.431817    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:13:26.512870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.512848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe9b18fc-ba30-48ce-b927-953c794ea102-config-file\") pod \"limitador-limitador-84994f7ffc-crfbz\" (UID: \"fe9b18fc-ba30-48ce-b927-953c794ea102\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.513316 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.512880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxlww\" (UniqueName: \"kubernetes.io/projected/fe9b18fc-ba30-48ce-b927-953c794ea102-kube-api-access-bxlww\") pod \"limitador-limitador-84994f7ffc-crfbz\" (UID: \"fe9b18fc-ba30-48ce-b927-953c794ea102\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.514115 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.514091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fe9b18fc-ba30-48ce-b927-953c794ea102-config-file\") pod \"limitador-limitador-84994f7ffc-crfbz\" (UID: \"fe9b18fc-ba30-48ce-b927-953c794ea102\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.524631 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.524608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxlww\" (UniqueName: \"kubernetes.io/projected/fe9b18fc-ba30-48ce-b927-953c794ea102-kube-api-access-bxlww\") pod \"limitador-limitador-84994f7ffc-crfbz\" (UID: \"fe9b18fc-ba30-48ce-b927-953c794ea102\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.604093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.604032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:26.906489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:26.906463    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:13:26.908009 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:13:26.907988    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe9b18fc_ba30_48ce_b927_953c794ea102.slice/crio-0d81cc61d93ee7b121d899363ed3f5b2d05d731495204155d7fb3b75d8d83980 WatchSource:0}: Error finding container 0d81cc61d93ee7b121d899363ed3f5b2d05d731495204155d7fb3b75d8d83980: Status 404 returned error can't find the container with id 0d81cc61d93ee7b121d899363ed3f5b2d05d731495204155d7fb3b75d8d83980
Sep 16 14:13:27.027519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:27.027488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:27.182519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:27.182455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:27.474475 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:27.474449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz" event={"ID":"fe9b18fc-ba30-48ce-b927-953c794ea102","Type":"ContainerStarted","Data":"0d81cc61d93ee7b121d899363ed3f5b2d05d731495204155d7fb3b75d8d83980"}
Sep 16 14:13:29.579455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:29.579422    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:29.650595 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:29.650563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:31.493339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:31.493302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz" event={"ID":"fe9b18fc-ba30-48ce-b927-953c794ea102","Type":"ContainerStarted","Data":"28c76a7d54fa6949afecd9c0c4c4596f021cb79138457b943d592d52e5e8a647"}
Sep 16 14:13:31.493751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:31.493423    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:31.513968 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:31.513927    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz" podStartSLOduration=1.288501658 podStartE2EDuration="5.513913988s" totalImagesPullingTime="4.22541233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:31.512683546 +0000 UTC m=+542.000693642" watchObservedRunningTime="2026-09-16 14:13:31.513913988 +0000 UTC m=+542.001924105"
Sep 16 14:13:32.513377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:32.513343    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:42.500202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:42.500173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-crfbz"
Sep 16 14:13:46.902622 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:46.902582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:46.992198 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:46.992161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:47.424698 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:47.424665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:13:50.665785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:13:50.665754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:00.074644 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.074605    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:00.193670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.193635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:00.546913 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.546884    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69585c4df8-mxhtz"]
Sep 16 14:14:00.549403 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.549384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:00.552487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.552467    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8h45d\""
Sep 16 14:14:00.553372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.553356    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 14:14:00.558862 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.558839    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69585c4df8-mxhtz"]
Sep 16 14:14:00.702442 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.702408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/915539ba-a56b-48f0-a69c-c1d9388d9124-tls-cert\") pod \"authorino-69585c4df8-mxhtz\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") " pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:00.702656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.702461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjg6z\" (UniqueName: \"kubernetes.io/projected/915539ba-a56b-48f0-a69c-c1d9388d9124-kube-api-access-vjg6z\") pod \"authorino-69585c4df8-mxhtz\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") " pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:00.803581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.803497    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/915539ba-a56b-48f0-a69c-c1d9388d9124-tls-cert\") pod \"authorino-69585c4df8-mxhtz\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") " pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:00.803581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.803547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjg6z\" (UniqueName: \"kubernetes.io/projected/915539ba-a56b-48f0-a69c-c1d9388d9124-kube-api-access-vjg6z\") pod \"authorino-69585c4df8-mxhtz\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") " pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:00.806014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.805989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/915539ba-a56b-48f0-a69c-c1d9388d9124-tls-cert\") pod \"authorino-69585c4df8-mxhtz\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") " pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:00.813880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.813859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjg6z\" (UniqueName: \"kubernetes.io/projected/915539ba-a56b-48f0-a69c-c1d9388d9124-kube-api-access-vjg6z\") pod \"authorino-69585c4df8-mxhtz\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") " pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:00.858688 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:00.858659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:14:01.140204 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:01.140180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69585c4df8-mxhtz"]
Sep 16 14:14:01.142656 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:14:01.142625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod915539ba_a56b_48f0_a69c_c1d9388d9124.slice/crio-6b3f92ed8fdd616bb7d7dc9fe26aa9423067096f0faeadd0128eecec117c260a WatchSource:0}: Error finding container 6b3f92ed8fdd616bb7d7dc9fe26aa9423067096f0faeadd0128eecec117c260a: Status 404 returned error can't find the container with id 6b3f92ed8fdd616bb7d7dc9fe26aa9423067096f0faeadd0128eecec117c260a
Sep 16 14:14:01.629946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:01.629913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69585c4df8-mxhtz" event={"ID":"915539ba-a56b-48f0-a69c-c1d9388d9124","Type":"ContainerStarted","Data":"6b3f92ed8fdd616bb7d7dc9fe26aa9423067096f0faeadd0128eecec117c260a"}
Sep 16 14:14:03.641558 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:03.641519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69585c4df8-mxhtz" event={"ID":"915539ba-a56b-48f0-a69c-c1d9388d9124","Type":"ContainerStarted","Data":"bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb"}
Sep 16 14:14:03.660841 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:03.660793    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69585c4df8-mxhtz" podStartSLOduration=2.052648012 podStartE2EDuration="3.660778705s" totalImagesPullingTime="1.608130693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:03.659414199 +0000 UTC m=+574.147424257" watchObservedRunningTime="2026-09-16 14:14:03.660778705 +0000 UTC m=+574.148788781"
Sep 16 14:14:08.597006 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:08.596968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:08.695462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:08.695421    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:08.795816 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:08.795779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:08.874614 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:08.874524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:08.971323 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:08.971290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:12.033772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:12.033737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:12.753915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:12.753879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:45.539945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.539910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:45.746896 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.746863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:45.819369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.819294    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"]
Sep 16 14:14:45.822404 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.822386    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:45.833274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.833246    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ccsnl\""
Sep 16 14:14:45.833414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.833258    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 14:14:45.838396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.838372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:14:45.847657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.847633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"]
Sep 16 14:14:45.873336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.873312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d0621e-1aa1-466a-a686-4c1c31d47f75-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-flpv7\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:45.873479 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.873354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lh5n\" (UniqueName: \"kubernetes.io/projected/e2d0621e-1aa1-466a-a686-4c1c31d47f75-kube-api-access-2lh5n\") pod \"payload-pre-processing-59fbc47bf8-flpv7\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:45.974572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.974535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d0621e-1aa1-466a-a686-4c1c31d47f75-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-flpv7\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:45.974754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.974587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lh5n\" (UniqueName: \"kubernetes.io/projected/e2d0621e-1aa1-466a-a686-4c1c31d47f75-kube-api-access-2lh5n\") pod \"payload-pre-processing-59fbc47bf8-flpv7\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:45.975313 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:45.975289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d0621e-1aa1-466a-a686-4c1c31d47f75-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-flpv7\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:46.000660 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:46.000631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lh5n\" (UniqueName: \"kubernetes.io/projected/e2d0621e-1aa1-466a-a686-4c1c31d47f75-kube-api-access-2lh5n\") pod \"payload-pre-processing-59fbc47bf8-flpv7\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:46.135760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:46.135681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:46.408955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:46.408926    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"]
Sep 16 14:14:46.411502 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:14:46.411465    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d0621e_1aa1_466a_a686_4c1c31d47f75.slice/crio-7a8f6372fda451cef747a56adce195c0f4a238ec91951a9decde64d6a058c1c0 WatchSource:0}: Error finding container 7a8f6372fda451cef747a56adce195c0f4a238ec91951a9decde64d6a058c1c0: Status 404 returned error can't find the container with id 7a8f6372fda451cef747a56adce195c0f4a238ec91951a9decde64d6a058c1c0
Sep 16 14:14:46.853751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:46.853718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7" event={"ID":"e2d0621e-1aa1-466a-a686-4c1c31d47f75","Type":"ContainerStarted","Data":"7a8f6372fda451cef747a56adce195c0f4a238ec91951a9decde64d6a058c1c0"}
Sep 16 14:14:50.873294 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:50.873258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7" event={"ID":"e2d0621e-1aa1-466a-a686-4c1c31d47f75","Type":"ContainerStarted","Data":"bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1"}
Sep 16 14:14:50.873653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:50.873324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:14:50.893039 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:50.892993    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7" podStartSLOduration=1.499512288 podStartE2EDuration="5.892975486s" totalImagesPullingTime="4.393463198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:50.891615732 +0000 UTC m=+621.379625809" watchObservedRunningTime="2026-09-16 14:14:50.892975486 +0000 UTC m=+621.380985562"
Sep 16 14:14:56.068256 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:56.068179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:14:58.302109 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:14:58.302076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:15:01.880918 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:01.880882    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:15:22.106283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:22.106251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:15:22.536939 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:22.536908    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:15:25.160661 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:25.160616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:15:39.915792 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:39.915747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:15:40.635224 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:40.635190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:15:46.802863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:46.802826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:15:47.873957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:47.873924    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:15:55.241306 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:55.241237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:15:56.401297 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:56.401247    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:15:57.520093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:57.520054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:15:57.680366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:15:57.680323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:16:02.881867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.881830    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"]
Sep 16 14:16:02.884574 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.884556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:02.888083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.888062    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 16 14:16:02.889109 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.889094    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 14:16:02.889206 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.889131    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 14:16:02.889424 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.889407    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k6tqt\""
Sep 16 14:16:02.902753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.902722    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"]
Sep 16 14:16:02.911596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:02.911562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:16:03.021528 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.021489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.021723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.021544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.021723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.021638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.021723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.021667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggg6c\" (UniqueName: \"kubernetes.io/projected/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-kube-api-access-ggg6c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.021894 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.021739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.021894 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.021799    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122305 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122305 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggg6c\" (UniqueName: \"kubernetes.io/projected/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-kube-api-access-ggg6c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.122895 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.122875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.124714 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.124688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.124844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.124827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.133877 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.133824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggg6c\" (UniqueName: \"kubernetes.io/projected/e1e19369-0cb3-4fe6-af0d-ad3d4593a492-kube-api-access-ggg6c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f\" (UID: \"e1e19369-0cb3-4fe6-af0d-ad3d4593a492\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.195586 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.195547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:03.476495 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.476463    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"]
Sep 16 14:16:03.479286 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:16:03.479251    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e19369_0cb3_4fe6_af0d_ad3d4593a492.slice/crio-2f6f3db4fc1c8f3730df58f22f3de4854a99ae01a0a93e6d07eedbad8c1a264e WatchSource:0}: Error finding container 2f6f3db4fc1c8f3730df58f22f3de4854a99ae01a0a93e6d07eedbad8c1a264e: Status 404 returned error can't find the container with id 2f6f3db4fc1c8f3730df58f22f3de4854a99ae01a0a93e6d07eedbad8c1a264e
Sep 16 14:16:03.480518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:03.480502    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:16:04.229489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:04.229446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f" event={"ID":"e1e19369-0cb3-4fe6-af0d-ad3d4593a492","Type":"ContainerStarted","Data":"2f6f3db4fc1c8f3730df58f22f3de4854a99ae01a0a93e6d07eedbad8c1a264e"}
Sep 16 14:16:04.391825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:04.391790    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:16:05.236296 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:05.236265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f" event={"ID":"e1e19369-0cb3-4fe6-af0d-ad3d4593a492","Type":"ContainerStarted","Data":"51c05e2629f3317cc72daf0f3e0277990d3693871e912279ec8916cecc5bcfb9"}
Sep 16 14:16:05.261468 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:05.261412    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f" podStartSLOduration=2.205231455 podStartE2EDuration="3.261394873s" totalImagesPullingTime="1.056163418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:16:05.260587688 +0000 UTC m=+695.748597763" watchObservedRunningTime="2026-09-16 14:16:05.261394873 +0000 UTC m=+695.749404947"
Sep 16 14:16:08.923710 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:08.923677    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:16:10.206489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:10.206453    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:16:13.195890 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:13.195842    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:13.196317 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:13.195916    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:13.208123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:13.208099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:13.285794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:13.285761    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f96c6f"
Sep 16 14:16:20.475877 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:20.475832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:16:22.678952 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:16:22.678872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:17:02.717665 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:02.717631    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69585c4df8-mxhtz"]
Sep 16 14:17:02.718270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:02.717835    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-69585c4df8-mxhtz" podUID="915539ba-a56b-48f0-a69c-c1d9388d9124" containerName="authorino" containerID="cri-o://bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb" gracePeriod=30
Sep 16 14:17:02.959642 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:02.959619    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:17:03.056243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.056160    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/915539ba-a56b-48f0-a69c-c1d9388d9124-tls-cert\") pod \"915539ba-a56b-48f0-a69c-c1d9388d9124\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") "
Sep 16 14:17:03.056398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.056318    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjg6z\" (UniqueName: \"kubernetes.io/projected/915539ba-a56b-48f0-a69c-c1d9388d9124-kube-api-access-vjg6z\") pod \"915539ba-a56b-48f0-a69c-c1d9388d9124\" (UID: \"915539ba-a56b-48f0-a69c-c1d9388d9124\") "
Sep 16 14:17:03.058287 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.058260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/915539ba-a56b-48f0-a69c-c1d9388d9124-kube-api-access-vjg6z" (OuterVolumeSpecName: "kube-api-access-vjg6z") pod "915539ba-a56b-48f0-a69c-c1d9388d9124" (UID: "915539ba-a56b-48f0-a69c-c1d9388d9124"). InnerVolumeSpecName "kube-api-access-vjg6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:17:03.067532 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.067510    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/915539ba-a56b-48f0-a69c-c1d9388d9124-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "915539ba-a56b-48f0-a69c-c1d9388d9124" (UID: "915539ba-a56b-48f0-a69c-c1d9388d9124"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:17:03.084009 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.083981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:03.157756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.157734    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjg6z\" (UniqueName: \"kubernetes.io/projected/915539ba-a56b-48f0-a69c-c1d9388d9124-kube-api-access-vjg6z\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.157756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.157755    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/915539ba-a56b-48f0-a69c-c1d9388d9124-tls-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.198984 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.198956    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"]
Sep 16 14:17:03.199189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.199162    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7" podUID="e2d0621e-1aa1-466a-a686-4c1c31d47f75" containerName="payload-pre-processing" containerID="cri-o://bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1" gracePeriod=30
Sep 16 14:17:03.229524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.229494    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:03.428893 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.428865    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:17:03.504871 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.504837    2580 generic.go:356] "Generic (PLEG): container finished" podID="915539ba-a56b-48f0-a69c-c1d9388d9124" containerID="bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb" exitCode=0
Sep 16 14:17:03.505026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.504913    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69585c4df8-mxhtz"
Sep 16 14:17:03.505026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.504924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69585c4df8-mxhtz" event={"ID":"915539ba-a56b-48f0-a69c-c1d9388d9124","Type":"ContainerDied","Data":"bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb"}
Sep 16 14:17:03.505026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.504973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69585c4df8-mxhtz" event={"ID":"915539ba-a56b-48f0-a69c-c1d9388d9124","Type":"ContainerDied","Data":"6b3f92ed8fdd616bb7d7dc9fe26aa9423067096f0faeadd0128eecec117c260a"}
Sep 16 14:17:03.505026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.504996    2580 scope.go:117] "RemoveContainer" containerID="bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb"
Sep 16 14:17:03.506354 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.506331    2580 generic.go:356] "Generic (PLEG): container finished" podID="e2d0621e-1aa1-466a-a686-4c1c31d47f75" containerID="bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1" exitCode=0
Sep 16 14:17:03.506481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.506374    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"
Sep 16 14:17:03.506481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.506408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7" event={"ID":"e2d0621e-1aa1-466a-a686-4c1c31d47f75","Type":"ContainerDied","Data":"bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1"}
Sep 16 14:17:03.506481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.506477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7" event={"ID":"e2d0621e-1aa1-466a-a686-4c1c31d47f75","Type":"ContainerDied","Data":"7a8f6372fda451cef747a56adce195c0f4a238ec91951a9decde64d6a058c1c0"}
Sep 16 14:17:03.515792 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.515773    2580 scope.go:117] "RemoveContainer" containerID="bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb"
Sep 16 14:17:03.516201 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:17:03.516175    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb\": container with ID starting with bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb not found: ID does not exist" containerID="bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb"
Sep 16 14:17:03.516299 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.516203    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb"} err="failed to get container status \"bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb\": rpc error: code = NotFound desc = could not find container \"bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb\": container with ID starting with bfda56a2e710488094cd60780796a49f615681e488c6859a9e5376e6dd95e1cb not found: ID does not exist"
Sep 16 14:17:03.516299 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.516220    2580 scope.go:117] "RemoveContainer" containerID="bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1"
Sep 16 14:17:03.523765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.523740    2580 scope.go:117] "RemoveContainer" containerID="bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1"
Sep 16 14:17:03.524017 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:17:03.524000    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1\": container with ID starting with bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1 not found: ID does not exist" containerID="bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1"
Sep 16 14:17:03.524069 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.524022    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1"} err="failed to get container status \"bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1\": rpc error: code = NotFound desc = could not find container \"bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1\": container with ID starting with bea73fee22f3112be1961e15857f9c566fe6b62f0a041658a54677662c15dde1 not found: ID does not exist"
Sep 16 14:17:03.531589 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.531572    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69585c4df8-mxhtz"]
Sep 16 14:17:03.535630 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.535613    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69585c4df8-mxhtz"]
Sep 16 14:17:03.561833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.561817    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d0621e-1aa1-466a-a686-4c1c31d47f75-plugins-config-volume\") pod \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") "
Sep 16 14:17:03.561908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.561866    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lh5n\" (UniqueName: \"kubernetes.io/projected/e2d0621e-1aa1-466a-a686-4c1c31d47f75-kube-api-access-2lh5n\") pod \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\" (UID: \"e2d0621e-1aa1-466a-a686-4c1c31d47f75\") "
Sep 16 14:17:03.562181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.562162    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d0621e-1aa1-466a-a686-4c1c31d47f75-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2d0621e-1aa1-466a-a686-4c1c31d47f75" (UID: "e2d0621e-1aa1-466a-a686-4c1c31d47f75"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:17:03.563902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.563884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d0621e-1aa1-466a-a686-4c1c31d47f75-kube-api-access-2lh5n" (OuterVolumeSpecName: "kube-api-access-2lh5n") pod "e2d0621e-1aa1-466a-a686-4c1c31d47f75" (UID: "e2d0621e-1aa1-466a-a686-4c1c31d47f75"). InnerVolumeSpecName "kube-api-access-2lh5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:17:03.663149 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.663079    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lh5n\" (UniqueName: \"kubernetes.io/projected/e2d0621e-1aa1-466a-a686-4c1c31d47f75-kube-api-access-2lh5n\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.663149 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.663103    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d0621e-1aa1-466a-a686-4c1c31d47f75-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.828116 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.828088    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"]
Sep 16 14:17:03.831279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:03.831255    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flpv7"]
Sep 16 14:17:04.005991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:04.005965    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="915539ba-a56b-48f0-a69c-c1d9388d9124" path="/var/lib/kubelet/pods/915539ba-a56b-48f0-a69c-c1d9388d9124/volumes"
Sep 16 14:17:04.006337 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:04.006322    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2d0621e-1aa1-466a-a686-4c1c31d47f75" path="/var/lib/kubelet/pods/e2d0621e-1aa1-466a-a686-4c1c31d47f75/volumes"
Sep 16 14:17:04.231275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:04.231237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:13.445672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:13.445623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:16.977124 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.977082    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"]
Sep 16 14:17:16.977496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.977452    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="915539ba-a56b-48f0-a69c-c1d9388d9124" containerName="authorino"
Sep 16 14:17:16.977496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.977463    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="915539ba-a56b-48f0-a69c-c1d9388d9124" containerName="authorino"
Sep 16 14:17:16.977496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.977483    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d0621e-1aa1-466a-a686-4c1c31d47f75" containerName="payload-pre-processing"
Sep 16 14:17:16.977496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.977488    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d0621e-1aa1-466a-a686-4c1c31d47f75" containerName="payload-pre-processing"
Sep 16 14:17:16.977622 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.977548    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2d0621e-1aa1-466a-a686-4c1c31d47f75" containerName="payload-pre-processing"
Sep 16 14:17:16.977622 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.977562    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="915539ba-a56b-48f0-a69c-c1d9388d9124" containerName="authorino"
Sep 16 14:17:16.982035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.982018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:16.984739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.984714    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Sep 16 14:17:16.984851 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.984719    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-9kh42\""
Sep 16 14:17:16.988585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:16.988565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"]
Sep 16 14:17:17.070478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:17.070452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmflh\" (UniqueName: \"kubernetes.io/projected/6157cc9c-3ae3-4628-aa56-d7c9edeae0c7-kube-api-access-tmflh\") pod \"ai-gateway-controller-7d7dbcd545-8kdtl\" (UID: \"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7\") " pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:17.171319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:17.171290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmflh\" (UniqueName: \"kubernetes.io/projected/6157cc9c-3ae3-4628-aa56-d7c9edeae0c7-kube-api-access-tmflh\") pod \"ai-gateway-controller-7d7dbcd545-8kdtl\" (UID: \"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7\") " pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:17.183530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:17.183508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmflh\" (UniqueName: \"kubernetes.io/projected/6157cc9c-3ae3-4628-aa56-d7c9edeae0c7-kube-api-access-tmflh\") pod \"ai-gateway-controller-7d7dbcd545-8kdtl\" (UID: \"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7\") " pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:17.294050 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:17.293983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:17.563884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:17.563822    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"]
Sep 16 14:17:17.566345 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:17:17.566314    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6157cc9c_3ae3_4628_aa56_d7c9edeae0c7.slice/crio-c40937dec3a0b66280268455a82430d9ee6124fc7c947c91db8e9a54f8fa2f89 WatchSource:0}: Error finding container c40937dec3a0b66280268455a82430d9ee6124fc7c947c91db8e9a54f8fa2f89: Status 404 returned error can't find the container with id c40937dec3a0b66280268455a82430d9ee6124fc7c947c91db8e9a54f8fa2f89
Sep 16 14:17:17.573359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:17.573326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl" event={"ID":"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7","Type":"ContainerStarted","Data":"c40937dec3a0b66280268455a82430d9ee6124fc7c947c91db8e9a54f8fa2f89"}
Sep 16 14:17:20.396612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.396539    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-28hd7"]
Sep 16 14:17:20.400024 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.400001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.402721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.402702    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ccsnl\""
Sep 16 14:17:20.402849 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.402724    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 14:17:20.412095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.412072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-28hd7"]
Sep 16 14:17:20.430550 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.430523    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"]
Sep 16 14:17:20.433943 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.433924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.438729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.438707    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:20.444030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.444008    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"]
Sep 16 14:17:20.502350 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.502325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e27dd0dd-1f25-4d76-84de-32602bba121c-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-28hd7\" (UID: \"e27dd0dd-1f25-4d76-84de-32602bba121c\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.502484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.502360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh5hg\" (UniqueName: \"kubernetes.io/projected/e27dd0dd-1f25-4d76-84de-32602bba121c-kube-api-access-dh5hg\") pod \"payload-processing-5d7dbbd56b-28hd7\" (UID: \"e27dd0dd-1f25-4d76-84de-32602bba121c\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.502484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.502400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71e9728-1ad3-4657-a332-91d21e89b914-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-wfbl9\" (UID: \"c71e9728-1ad3-4657-a332-91d21e89b914\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.502484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.502446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztc7n\" (UniqueName: \"kubernetes.io/projected/c71e9728-1ad3-4657-a332-91d21e89b914-kube-api-access-ztc7n\") pod \"payload-pre-processing-6cd864bc95-wfbl9\" (UID: \"c71e9728-1ad3-4657-a332-91d21e89b914\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.535754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.535733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:20.590486 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.590464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl" event={"ID":"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7","Type":"ContainerStarted","Data":"dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c"}
Sep 16 14:17:20.590596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.590567    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:20.603671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.603650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e27dd0dd-1f25-4d76-84de-32602bba121c-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-28hd7\" (UID: \"e27dd0dd-1f25-4d76-84de-32602bba121c\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.603740 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.603675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh5hg\" (UniqueName: \"kubernetes.io/projected/e27dd0dd-1f25-4d76-84de-32602bba121c-kube-api-access-dh5hg\") pod \"payload-processing-5d7dbbd56b-28hd7\" (UID: \"e27dd0dd-1f25-4d76-84de-32602bba121c\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.603740 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.603705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71e9728-1ad3-4657-a332-91d21e89b914-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-wfbl9\" (UID: \"c71e9728-1ad3-4657-a332-91d21e89b914\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.603740 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.603728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztc7n\" (UniqueName: \"kubernetes.io/projected/c71e9728-1ad3-4657-a332-91d21e89b914-kube-api-access-ztc7n\") pod \"payload-pre-processing-6cd864bc95-wfbl9\" (UID: \"c71e9728-1ad3-4657-a332-91d21e89b914\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.604511 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.604496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71e9728-1ad3-4657-a332-91d21e89b914-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-wfbl9\" (UID: \"c71e9728-1ad3-4657-a332-91d21e89b914\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.604552 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.604518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e27dd0dd-1f25-4d76-84de-32602bba121c-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-28hd7\" (UID: \"e27dd0dd-1f25-4d76-84de-32602bba121c\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.607508 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.607471    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl" podStartSLOduration=2.192106735 podStartE2EDuration="4.607459414s" totalImagesPullingTime="2.415352679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:20.606452794 +0000 UTC m=+771.094462870" watchObservedRunningTime="2026-09-16 14:17:20.607459414 +0000 UTC m=+771.095469481"
Sep 16 14:17:20.612002 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.611973    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh5hg\" (UniqueName: \"kubernetes.io/projected/e27dd0dd-1f25-4d76-84de-32602bba121c-kube-api-access-dh5hg\") pod \"payload-processing-5d7dbbd56b-28hd7\" (UID: \"e27dd0dd-1f25-4d76-84de-32602bba121c\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.612102 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.611973    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztc7n\" (UniqueName: \"kubernetes.io/projected/c71e9728-1ad3-4657-a332-91d21e89b914-kube-api-access-ztc7n\") pod \"payload-pre-processing-6cd864bc95-wfbl9\" (UID: \"c71e9728-1ad3-4657-a332-91d21e89b914\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.711639 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.711613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:20.746819 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.746793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:20.985434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:20.985409    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-28hd7"]
Sep 16 14:17:20.987602 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:17:20.987574    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27dd0dd_1f25_4d76_84de_32602bba121c.slice/crio-7e9dcdd840b249f5f48f800018c3cd7466c64493b2cc585a03c1b89430b6cbb8 WatchSource:0}: Error finding container 7e9dcdd840b249f5f48f800018c3cd7466c64493b2cc585a03c1b89430b6cbb8: Status 404 returned error can't find the container with id 7e9dcdd840b249f5f48f800018c3cd7466c64493b2cc585a03c1b89430b6cbb8
Sep 16 14:17:21.025888 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:21.025866    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"]
Sep 16 14:17:21.028041 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:17:21.028010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71e9728_1ad3_4657_a332_91d21e89b914.slice/crio-e89b3a3fa136daaa20b4d41b066c9c7c2046b7da5f8fbd8c7c42c5809f8e352b WatchSource:0}: Error finding container e89b3a3fa136daaa20b4d41b066c9c7c2046b7da5f8fbd8c7c42c5809f8e352b: Status 404 returned error can't find the container with id e89b3a3fa136daaa20b4d41b066c9c7c2046b7da5f8fbd8c7c42c5809f8e352b
Sep 16 14:17:21.596048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:21.596011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" event={"ID":"e27dd0dd-1f25-4d76-84de-32602bba121c","Type":"ContainerStarted","Data":"7e9dcdd840b249f5f48f800018c3cd7466c64493b2cc585a03c1b89430b6cbb8"}
Sep 16 14:17:21.597250 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:21.597226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" event={"ID":"c71e9728-1ad3-4657-a332-91d21e89b914","Type":"ContainerStarted","Data":"e89b3a3fa136daaa20b4d41b066c9c7c2046b7da5f8fbd8c7c42c5809f8e352b"}
Sep 16 14:17:22.607563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:22.607475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" event={"ID":"c71e9728-1ad3-4657-a332-91d21e89b914","Type":"ContainerStarted","Data":"ef09f0876ed38a8d3b53d144a32787cec7afc92c37dda9035c841b2e8468fdfe"}
Sep 16 14:17:22.607906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:22.607677    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:22.608953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:22.608931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" event={"ID":"e27dd0dd-1f25-4d76-84de-32602bba121c","Type":"ContainerStarted","Data":"dc47c4117bbe15d60654979469e3696db9542806b4a407c61247e2c8300a3b8d"}
Sep 16 14:17:22.609077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:22.609060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:22.622929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:22.622886    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" podStartSLOduration=1.305883919 podStartE2EDuration="2.622873394s" totalImagesPullingTime="1.316989475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:22.62204833 +0000 UTC m=+773.110058407" watchObservedRunningTime="2026-09-16 14:17:22.622873394 +0000 UTC m=+773.110883488"
Sep 16 14:17:22.636915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:22.636878    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" podStartSLOduration=1.2808536 podStartE2EDuration="2.636865813s" totalImagesPullingTime="1.356012213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:22.636822646 +0000 UTC m=+773.124832724" watchObservedRunningTime="2026-09-16 14:17:22.636865813 +0000 UTC m=+773.124875916"
Sep 16 14:17:26.910698 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:26.910659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:27.011684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:27.011646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:31.599686 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:31.599652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:31.798691 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:31.798659    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"]
Sep 16 14:17:31.803440 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:31.803420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"
Sep 16 14:17:31.813716 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:31.813695    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"]
Sep 16 14:17:31.896088 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:31.896034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtbdl\" (UniqueName: \"kubernetes.io/projected/f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea-kube-api-access-vtbdl\") pod \"ai-gateway-controller-6f4fb96fc-phgkb\" (UID: \"f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea\") " pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"
Sep 16 14:17:31.996862 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:31.996839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtbdl\" (UniqueName: \"kubernetes.io/projected/f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea-kube-api-access-vtbdl\") pod \"ai-gateway-controller-6f4fb96fc-phgkb\" (UID: \"f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea\") " pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"
Sep 16 14:17:32.015288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:32.015262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtbdl\" (UniqueName: \"kubernetes.io/projected/f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea-kube-api-access-vtbdl\") pod \"ai-gateway-controller-6f4fb96fc-phgkb\" (UID: \"f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea\") " pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"
Sep 16 14:17:32.113194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:32.113173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"
Sep 16 14:17:32.380739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:32.380684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"]
Sep 16 14:17:32.382795 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:17:32.382768    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ce0a0c_9cdc_4ccf_bf82_82f18536f6ea.slice/crio-2d0b22b6874c810e34283bd12b9ba2bfcb1ebd360859eec59b7c2a1044f13b1a WatchSource:0}: Error finding container 2d0b22b6874c810e34283bd12b9ba2bfcb1ebd360859eec59b7c2a1044f13b1a: Status 404 returned error can't find the container with id 2d0b22b6874c810e34283bd12b9ba2bfcb1ebd360859eec59b7c2a1044f13b1a
Sep 16 14:17:32.667203 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:32.667167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb" event={"ID":"f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea","Type":"ContainerStarted","Data":"2d0b22b6874c810e34283bd12b9ba2bfcb1ebd360859eec59b7c2a1044f13b1a"}
Sep 16 14:17:33.616783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:33.616753    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9"
Sep 16 14:17:33.617057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:33.616822    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7"
Sep 16 14:17:33.617175 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:33.617158    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:17:33.617355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:33.617332    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:17:33.674106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:33.674077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb" event={"ID":"f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea","Type":"ContainerStarted","Data":"88083cc6094917001a104caf96166cf1a582512dd9f7c05835055ddf22dd0796"}
Sep 16 14:17:33.674501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:33.674313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"
Sep 16 14:17:33.690183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:33.690128    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb" podStartSLOduration=2.103467592 podStartE2EDuration="2.690116774s" totalImagesPullingTime="586.649182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:33.689084457 +0000 UTC m=+784.177094534" watchObservedRunningTime="2026-09-16 14:17:33.690116774 +0000 UTC m=+784.178126876"
Sep 16 14:17:44.681280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:44.681245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6f4fb96fc-phgkb"
Sep 16 14:17:44.726281 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:44.726247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"]
Sep 16 14:17:44.726446 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:44.726419    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl" podUID="6157cc9c-3ae3-4628-aa56-d7c9edeae0c7" containerName="manager" containerID="cri-o://dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c" gracePeriod=30
Sep 16 14:17:44.971880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:44.971860    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:45.107844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.107818    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmflh\" (UniqueName: \"kubernetes.io/projected/6157cc9c-3ae3-4628-aa56-d7c9edeae0c7-kube-api-access-tmflh\") pod \"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7\" (UID: \"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7\") "
Sep 16 14:17:45.109856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.109821    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6157cc9c-3ae3-4628-aa56-d7c9edeae0c7-kube-api-access-tmflh" (OuterVolumeSpecName: "kube-api-access-tmflh") pod "6157cc9c-3ae3-4628-aa56-d7c9edeae0c7" (UID: "6157cc9c-3ae3-4628-aa56-d7c9edeae0c7"). InnerVolumeSpecName "kube-api-access-tmflh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:17:45.209316 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.209291    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmflh\" (UniqueName: \"kubernetes.io/projected/6157cc9c-3ae3-4628-aa56-d7c9edeae0c7-kube-api-access-tmflh\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:17:45.737668 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.737635    2580 generic.go:356] "Generic (PLEG): container finished" podID="6157cc9c-3ae3-4628-aa56-d7c9edeae0c7" containerID="dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c" exitCode=0
Sep 16 14:17:45.738035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.737683    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"
Sep 16 14:17:45.738035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.737729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl" event={"ID":"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7","Type":"ContainerDied","Data":"dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c"}
Sep 16 14:17:45.738035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.737786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl" event={"ID":"6157cc9c-3ae3-4628-aa56-d7c9edeae0c7","Type":"ContainerDied","Data":"c40937dec3a0b66280268455a82430d9ee6124fc7c947c91db8e9a54f8fa2f89"}
Sep 16 14:17:45.738035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.737809    2580 scope.go:117] "RemoveContainer" containerID="dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c"
Sep 16 14:17:45.747049 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.747027    2580 scope.go:117] "RemoveContainer" containerID="dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c"
Sep 16 14:17:45.747371 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:17:45.747331    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c\": container with ID starting with dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c not found: ID does not exist" containerID="dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c"
Sep 16 14:17:45.747371 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.747360    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c"} err="failed to get container status \"dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c\": rpc error: code = NotFound desc = could not find container \"dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c\": container with ID starting with dd43714407ef657fcec3b298573a033d1e2a9a5c71236fbb1bc5c877d1ab2a0c not found: ID does not exist"
Sep 16 14:17:45.759663 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.759638    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"]
Sep 16 14:17:45.765007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:45.764984    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-8kdtl"]
Sep 16 14:17:46.006047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:46.005987    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6157cc9c-3ae3-4628-aa56-d7c9edeae0c7" path="/var/lib/kubelet/pods/6157cc9c-3ae3-4628-aa56-d7c9edeae0c7/volumes"
Sep 16 14:17:46.008856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:46.008831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:46.357301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:46.357233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:47.505444 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:47.505412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:47.598184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:47.598129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:48.651066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:48.651033    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:17:48.754330 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:17:48.754301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:04.888971 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:04.888886    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:04.990947 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:04.990909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:05.353937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.353908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:05.525309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.525275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:05.534227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.534205    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"]
Sep 16 14:18:05.534619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.534606    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6157cc9c-3ae3-4628-aa56-d7c9edeae0c7" containerName="manager"
Sep 16 14:18:05.534668 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.534620    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6157cc9c-3ae3-4628-aa56-d7c9edeae0c7" containerName="manager"
Sep 16 14:18:05.534705 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.534686    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6157cc9c-3ae3-4628-aa56-d7c9edeae0c7" containerName="manager"
Sep 16 14:18:05.537863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.537845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.540750 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.540730    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 14:18:05.540983 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.540957    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e24c87-openshift-default-dockercfg-5bfxq\""
Sep 16 14:18:05.579423 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.579404    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"]
Sep 16 14:18:05.664184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-istio-token\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72f083bc-289b-4da9-92e6-cde78275a52f-istiod-ca-cert\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-socket\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-data\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-envoy\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx2hb\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-kube-api-access-tx2hb\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-certs\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72f083bc-289b-4da9-92e6-cde78275a52f-istio-podinfo\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.664572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.664373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-credential-socket\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.764833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.764809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2hb\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-kube-api-access-tx2hb\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.764941 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.764864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-certs\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.764941 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.764913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72f083bc-289b-4da9-92e6-cde78275a52f-istio-podinfo\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.764940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-credential-socket\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.764992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-istio-token\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72f083bc-289b-4da9-92e6-cde78275a52f-istiod-ca-cert\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-socket\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:05.765237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-data\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-envoy\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765449 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-certs\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765520 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-credential-socket\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765520 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-socket\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765620 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-data\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.765802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.765782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72f083bc-289b-4da9-92e6-cde78275a52f-istiod-ca-cert\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.767388 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.767369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-envoy\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.767468 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.767433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72f083bc-289b-4da9-92e6-cde78275a52f-istio-podinfo\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.807361 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.807339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-istio-token\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.812859 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.812838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2hb\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-kube-api-access-tx2hb\") pod \"e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") " pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:05.848263 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:05.848242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:06.029222 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:06.029195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:06.163573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:06.163549    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"]
Sep 16 14:18:06.166007 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:06.165981    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f083bc_289b_4da9_92e6_cde78275a52f.slice/crio-73bf708bc724d943ff64366a5666fe1f6ae9f5d8faec2e96bcffcbaf7f9ef0a1 WatchSource:0}: Error finding container 73bf708bc724d943ff64366a5666fe1f6ae9f5d8faec2e96bcffcbaf7f9ef0a1: Status 404 returned error can't find the container with id 73bf708bc724d943ff64366a5666fe1f6ae9f5d8faec2e96bcffcbaf7f9ef0a1
Sep 16 14:18:06.843627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:06.843594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" event={"ID":"72f083bc-289b-4da9-92e6-cde78275a52f","Type":"ContainerStarted","Data":"73bf708bc724d943ff64366a5666fe1f6ae9f5d8faec2e96bcffcbaf7f9ef0a1"}
Sep 16 14:18:09.144059 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.144014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:09.248521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.248487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:09.298700 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.298663    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:09.298811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.298734    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:09.298811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.298762    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:09.401547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.401460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:09.860153 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.860117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" event={"ID":"72f083bc-289b-4da9-92e6-cde78275a52f","Type":"ContainerStarted","Data":"e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53"}
Sep 16 14:18:09.912417 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:09.912372    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podStartSLOduration=1.781694887 podStartE2EDuration="4.912356508s" totalImagesPullingTime="3.130661621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:09.911414338 +0000 UTC m=+820.399424431" watchObservedRunningTime="2026-09-16 14:18:09.912356508 +0000 UTC m=+820.400366584"
Sep 16 14:18:10.557743 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:10.557712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:10.849330 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:10.849265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:10.853669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:10.853647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:10.864431 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:10.864409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:10.864963 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:10.864946    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:10.867711 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:10.867690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:10.968127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:10.968100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:11.224201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.224169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:11.331797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.331771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:11.426317 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.426290    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"]
Sep 16 14:18:11.429876 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.429859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.433702 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.433684    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-2ad565\""
Sep 16 14:18:11.454354 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.454336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"]
Sep 16 14:18:11.519577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.519520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.519577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.519570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbrdg\" (UniqueName: \"kubernetes.io/projected/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-kube-api-access-nbrdg\") pod \"payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.538262 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.538234    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"]
Sep 16 14:18:11.541739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.541720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.545939 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.545924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-2ad565-dockercfg-462cn\""
Sep 16 14:18:11.553695 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.553675    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"]
Sep 16 14:18:11.620773 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.620748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f98486f-75e8-4bb6-9e7d-842a485268f4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.621076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.620792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbrdg\" (UniqueName: \"kubernetes.io/projected/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-kube-api-access-nbrdg\") pod \"payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.621076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.620846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cddgj\" (UniqueName: \"kubernetes.io/projected/9f98486f-75e8-4bb6-9e7d-842a485268f4-kube-api-access-cddgj\") pod \"payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.621076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.620869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.621575 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.621556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.629248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.629229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbrdg\" (UniqueName: \"kubernetes.io/projected/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-kube-api-access-nbrdg\") pod \"payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.721433 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.721409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f98486f-75e8-4bb6-9e7d-842a485268f4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.721521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.721484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cddgj\" (UniqueName: \"kubernetes.io/projected/9f98486f-75e8-4bb6-9e7d-842a485268f4-kube-api-access-cddgj\") pod \"payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.722014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.721997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f98486f-75e8-4bb6-9e7d-842a485268f4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.730366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.730347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cddgj\" (UniqueName: \"kubernetes.io/projected/9f98486f-75e8-4bb6-9e7d-842a485268f4-kube-api-access-cddgj\") pod \"payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.740563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.740540    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:11.851402 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.851376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:11.952806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:11.952777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:12.015746 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.015476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"]
Sep 16 14:18:12.017950 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:12.017921    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e05fdf3_4dab_4b4e_ae6f_bd37d0b37b4d.slice/crio-6acad29cb4932d7f8e270097e9669087d2a2d41c06b2063f05fa523a8f5ecd04 WatchSource:0}: Error finding container 6acad29cb4932d7f8e270097e9669087d2a2d41c06b2063f05fa523a8f5ecd04: Status 404 returned error can't find the container with id 6acad29cb4932d7f8e270097e9669087d2a2d41c06b2063f05fa523a8f5ecd04
Sep 16 14:18:12.069750 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.069724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:12.126455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.126434    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"]
Sep 16 14:18:12.128115 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:12.128092    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f98486f_75e8_4bb6_9e7d_842a485268f4.slice/crio-70d8ddb86e4ba2af5704686c42cfa4610fa72a0f3de01b8f99222856718893a0 WatchSource:0}: Error finding container 70d8ddb86e4ba2af5704686c42cfa4610fa72a0f3de01b8f99222856718893a0: Status 404 returned error can't find the container with id 70d8ddb86e4ba2af5704686c42cfa4610fa72a0f3de01b8f99222856718893a0
Sep 16 14:18:12.566064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.566016    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:12.695834 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.695795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:12.764842 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.764815    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"]
Sep 16 14:18:12.768449 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.768436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:12.771104 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.771084    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-08475e\""
Sep 16 14:18:12.779891 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.779869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"]
Sep 16 14:18:12.781441 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.781413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:12.829247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.829195    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"]
Sep 16 14:18:12.830920 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.830899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:12.831015 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.830966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxlc\" (UniqueName: \"kubernetes.io/projected/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-kube-api-access-gxxlc\") pod \"payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:12.832600 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.832584    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:12.835025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.835010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-08475e-dockercfg-76cdh\""
Sep 16 14:18:12.842573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.842553    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"]
Sep 16 14:18:12.877230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.877199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn" event={"ID":"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d","Type":"ContainerStarted","Data":"9bf3e7fdd1c30c30ae72c612b368a45cfb81ac674e938c0c7533f7d6c0b5a13a"}
Sep 16 14:18:12.877325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.877249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn" event={"ID":"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d","Type":"ContainerStarted","Data":"6acad29cb4932d7f8e270097e9669087d2a2d41c06b2063f05fa523a8f5ecd04"}
Sep 16 14:18:12.877325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.877279    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:12.878812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.878789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp" event={"ID":"9f98486f-75e8-4bb6-9e7d-842a485268f4","Type":"ContainerStarted","Data":"eaf1418c99d2c3dc0d46e55dc15755f8b8f99a3d27e1ed420fbf843015424270"}
Sep 16 14:18:12.878891 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.878835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp" event={"ID":"9f98486f-75e8-4bb6-9e7d-842a485268f4","Type":"ContainerStarted","Data":"70d8ddb86e4ba2af5704686c42cfa4610fa72a0f3de01b8f99222856718893a0"}
Sep 16 14:18:12.878891 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.878868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:12.895927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.895888    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn" podStartSLOduration=1.89587489 podStartE2EDuration="1.89587489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:12.894034027 +0000 UTC m=+823.382044105" watchObservedRunningTime="2026-09-16 14:18:12.89587489 +0000 UTC m=+823.383884965"
Sep 16 14:18:12.914722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.914681    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp" podStartSLOduration=1.9146696699999999 podStartE2EDuration="1.91466967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:12.912632681 +0000 UTC m=+823.400642753" watchObservedRunningTime="2026-09-16 14:18:12.91466967 +0000 UTC m=+823.402679747"
Sep 16 14:18:12.932012 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.931991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxlc\" (UniqueName: \"kubernetes.io/projected/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-kube-api-access-gxxlc\") pod \"payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:12.932398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.932375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/436fef94-26bf-4af1-8b9c-22dfa97c6d53-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:12.932632 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.932613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:12.932722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.932705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gggm\" (UniqueName: \"kubernetes.io/projected/436fef94-26bf-4af1-8b9c-22dfa97c6d53-kube-api-access-9gggm\") pod \"payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:12.933215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.933197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:12.940509 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:12.940490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxlc\" (UniqueName: \"kubernetes.io/projected/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-kube-api-access-gxxlc\") pod \"payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:13.033285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.033261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gggm\" (UniqueName: \"kubernetes.io/projected/436fef94-26bf-4af1-8b9c-22dfa97c6d53-kube-api-access-9gggm\") pod \"payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:13.033391 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.033327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/436fef94-26bf-4af1-8b9c-22dfa97c6d53-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:13.033930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.033914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/436fef94-26bf-4af1-8b9c-22dfa97c6d53-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:13.041230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.041208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gggm\" (UniqueName: \"kubernetes.io/projected/436fef94-26bf-4af1-8b9c-22dfa97c6d53-kube-api-access-9gggm\") pod \"payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:13.080050 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.080001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:13.144976 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.144947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:13.354089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.354066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"]
Sep 16 14:18:13.355879 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:13.355852    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4434d2a7_1c12_4eaa_902c_ef5f3a0a35da.slice/crio-cb8f3b9a2ae99f5be7c272241a52a2d7660a9b1976b4d6d6eb2350bb0e57747f WatchSource:0}: Error finding container cb8f3b9a2ae99f5be7c272241a52a2d7660a9b1976b4d6d6eb2350bb0e57747f: Status 404 returned error can't find the container with id cb8f3b9a2ae99f5be7c272241a52a2d7660a9b1976b4d6d6eb2350bb0e57747f
Sep 16 14:18:13.426683 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.426658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"]
Sep 16 14:18:13.427964 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:13.427937    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436fef94_26bf_4af1_8b9c_22dfa97c6d53.slice/crio-b1f288c3167f51fa8d4e72b218e5bd4259ee26c683042faff1a83495867629c1 WatchSource:0}: Error finding container b1f288c3167f51fa8d4e72b218e5bd4259ee26c683042faff1a83495867629c1: Status 404 returned error can't find the container with id b1f288c3167f51fa8d4e72b218e5bd4259ee26c683042faff1a83495867629c1
Sep 16 14:18:13.445272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.445243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:13.885771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.885739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929" event={"ID":"436fef94-26bf-4af1-8b9c-22dfa97c6d53","Type":"ContainerStarted","Data":"05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c"}
Sep 16 14:18:13.886191 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.885786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929" event={"ID":"436fef94-26bf-4af1-8b9c-22dfa97c6d53","Type":"ContainerStarted","Data":"b1f288c3167f51fa8d4e72b218e5bd4259ee26c683042faff1a83495867629c1"}
Sep 16 14:18:13.886191 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.885862    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:13.887578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.887556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc" event={"ID":"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da","Type":"ContainerStarted","Data":"8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172"}
Sep 16 14:18:13.887711 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.887594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc" event={"ID":"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da","Type":"ContainerStarted","Data":"cb8f3b9a2ae99f5be7c272241a52a2d7660a9b1976b4d6d6eb2350bb0e57747f"}
Sep 16 14:18:13.887781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.887718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:13.903309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.903272    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929" podStartSLOduration=1.9032600610000001 podStartE2EDuration="1.903260061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:13.902162214 +0000 UTC m=+824.390172289" watchObservedRunningTime="2026-09-16 14:18:13.903260061 +0000 UTC m=+824.391270135"
Sep 16 14:18:13.921222 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:13.921187    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc" podStartSLOduration=1.921176815 podStartE2EDuration="1.921176815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:13.920077997 +0000 UTC m=+824.408088085" watchObservedRunningTime="2026-09-16 14:18:13.921176815 +0000 UTC m=+824.409186891"
Sep 16 14:18:14.317462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:14.317433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:15.228449 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.228420    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:15.329270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.329238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:15.522437 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.522368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:15.804299 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.804216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:15.924775 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.924744    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"]
Sep 16 14:18:15.928241 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.928226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:15.931218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.931197    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e24c87\""
Sep 16 14:18:15.931332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.931256    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e24c87-dockercfg-k42jb\""
Sep 16 14:18:15.944612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.944590    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"]
Sep 16 14:18:15.961302 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.961280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e70d8e47-235f-4205-bf50-33b8226bb575-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:15.961394 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:15.961327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzgbg\" (UniqueName: \"kubernetes.io/projected/e70d8e47-235f-4205-bf50-33b8226bb575-kube-api-access-kzgbg\") pod \"payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:16.062484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.062398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e70d8e47-235f-4205-bf50-33b8226bb575-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:16.062484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.062464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzgbg\" (UniqueName: \"kubernetes.io/projected/e70d8e47-235f-4205-bf50-33b8226bb575-kube-api-access-kzgbg\") pod \"payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:16.063165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.063125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e70d8e47-235f-4205-bf50-33b8226bb575-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:16.075003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.074978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzgbg\" (UniqueName: \"kubernetes.io/projected/e70d8e47-235f-4205-bf50-33b8226bb575-kube-api-access-kzgbg\") pod \"payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:16.239593 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.239564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:16.360568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.360496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:16.527807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.527782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"]
Sep 16 14:18:16.529970 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:16.529944    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70d8e47_235f_4205_bf50_33b8226bb575.slice/crio-7809be8145f9905ce8b95059f784b4fbe2a4f00c645241f77b3a6789532821c2 WatchSource:0}: Error finding container 7809be8145f9905ce8b95059f784b4fbe2a4f00c645241f77b3a6789532821c2: Status 404 returned error can't find the container with id 7809be8145f9905ce8b95059f784b4fbe2a4f00c645241f77b3a6789532821c2
Sep 16 14:18:16.903349 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.903317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz" event={"ID":"e70d8e47-235f-4205-bf50-33b8226bb575","Type":"ContainerStarted","Data":"6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a"}
Sep 16 14:18:16.903525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.903369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz" event={"ID":"e70d8e47-235f-4205-bf50-33b8226bb575","Type":"ContainerStarted","Data":"7809be8145f9905ce8b95059f784b4fbe2a4f00c645241f77b3a6789532821c2"}
Sep 16 14:18:16.903525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:16.903391    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:18.011524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:18.011467    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz" podStartSLOduration=3.011446962 podStartE2EDuration="3.011446962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:16.927761399 +0000 UTC m=+827.415771479" watchObservedRunningTime="2026-09-16 14:18:18.011446962 +0000 UTC m=+828.499457039"
Sep 16 14:18:18.012421 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:18.012403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:18:23.890307 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:23.890273    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:18:23.890654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:23.890348    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:18:24.893951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:24.893921    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:18:24.894354 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:24.893991    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:18:25.016181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:25.016123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:25.188513 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:25.188425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:25.418776 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:25.418742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:26.906317 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:26.906273    2580 patch_prober.go:28] interesting pod/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 14:18:26.906706 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:26.906339    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 14:18:27.910023 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:27.909994    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:18:28.966569 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:28.966538    2580 generic.go:356] "Generic (PLEG): container finished" podID="72f083bc-289b-4da9-92e6-cde78275a52f" containerID="e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53" exitCode=0
Sep 16 14:18:28.966987 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:28.966625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" event={"ID":"72f083bc-289b-4da9-92e6-cde78275a52f","Type":"ContainerDied","Data":"e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53"}
Sep 16 14:18:28.967086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:28.967069    2580 scope.go:117] "RemoveContainer" containerID="e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53"
Sep 16 14:18:28.969386 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:28.969341    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:28.969514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:28.969480    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:28.969590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:28.969553    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:29.973750 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:29.973721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" event={"ID":"72f083bc-289b-4da9-92e6-cde78275a52f","Type":"ContainerStarted","Data":"c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f"}
Sep 16 14:18:30.848827 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:30.848791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:30.849028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:30.849006    2580 patch_prober.go:28] interesting pod/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 16 14:18:30.849096 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:30.849065    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 16 14:18:31.401328 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:31.401243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:31.539392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:31.539360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:31.650652 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:31.650616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:31.762440 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:31.762408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:31.848921 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:31.848889    2580 patch_prober.go:28] interesting pod/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 16 14:18:31.849078 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:31.848959    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 16 14:18:31.862355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:31.862314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:32.849336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:32.849298    2580 patch_prober.go:28] interesting pod/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 16 14:18:32.849797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:32.849359    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 16 14:18:33.849621 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:33.849581    2580 patch_prober.go:28] interesting pod/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 16 14:18:33.850070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:33.849644    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 16 14:18:34.033753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:34.033720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:34.135524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:34.135443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:34.493812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:34.493772    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:34.643209 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:34.643171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:34.770866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:34.770789    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:34.849176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:34.849124    2580 patch_prober.go:28] interesting pod/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 16 14:18:34.849332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:34.849209    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 16 14:18:35.388801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:35.388771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:35.851822 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:35.851796    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:35.852068 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:35.852047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:35.852806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:35.852791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:18:36.528306 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:36.528274    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:36.934199 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:36.934098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:37.262941 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:37.262910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:37.544152 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:37.544066    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:37.700166 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:37.700120    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:38.079981 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:38.079946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:38.317064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:38.317034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:38.842132 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:38.842096    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:38.938874 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:38.938842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:39.264520 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:39.264486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:39.363725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:39.363692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:39.761319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:39.761285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:39.861922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:39.861891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:41.000370 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:41.000333    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:18:43.028309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:43.028272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:43.127426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:43.127393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:43.336413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:43.336313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:44.293526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:44.293489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:45.345932 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:45.345891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:46.226161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.226108    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:46.305464 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.305436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:46.691344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.691261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:46.839368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.839332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:46.933835 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.933802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:46.960545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.960522    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"]
Sep 16 14:18:46.968055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.968035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:46.971269 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.971250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-46e82d-dockercfg-cb2kv\""
Sep 16 14:18:46.971732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.971713    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-46e82d\""
Sep 16 14:18:46.976786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:46.976766    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"]
Sep 16 14:18:47.062564 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.062535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32f3ec55-6797-4167-aa9f-c4b7748c36dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:47.062699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.062673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfq62\" (UniqueName: \"kubernetes.io/projected/32f3ec55-6797-4167-aa9f-c4b7748c36dd-kube-api-access-dfq62\") pod \"payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:47.163545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.163519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32f3ec55-6797-4167-aa9f-c4b7748c36dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:47.163690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.163578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfq62\" (UniqueName: \"kubernetes.io/projected/32f3ec55-6797-4167-aa9f-c4b7748c36dd-kube-api-access-dfq62\") pod \"payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:47.164184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.164168    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32f3ec55-6797-4167-aa9f-c4b7748c36dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:47.174044 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.174017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfq62\" (UniqueName: \"kubernetes.io/projected/32f3ec55-6797-4167-aa9f-c4b7748c36dd-kube-api-access-dfq62\") pod \"payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:47.280298 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.280219    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:47.339102 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.339063    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:47.553611 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:47.553589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"]
Sep 16 14:18:47.555575 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:47.555547    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f3ec55_6797_4167_aa9f_c4b7748c36dd.slice/crio-4c7ba78989c3398901d2510a179a1b82d2daa823856c72c36c44688aca62cc82 WatchSource:0}: Error finding container 4c7ba78989c3398901d2510a179a1b82d2daa823856c72c36c44688aca62cc82: Status 404 returned error can't find the container with id 4c7ba78989c3398901d2510a179a1b82d2daa823856c72c36c44688aca62cc82
Sep 16 14:18:48.073352 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:48.073317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv" event={"ID":"32f3ec55-6797-4167-aa9f-c4b7748c36dd","Type":"ContainerStarted","Data":"b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878"}
Sep 16 14:18:48.073732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:48.073372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv" event={"ID":"32f3ec55-6797-4167-aa9f-c4b7748c36dd","Type":"ContainerStarted","Data":"4c7ba78989c3398901d2510a179a1b82d2daa823856c72c36c44688aca62cc82"}
Sep 16 14:18:48.073732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:48.073465    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:48.091813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:48.091768    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv" podStartSLOduration=2.091751542 podStartE2EDuration="2.091751542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:48.090701548 +0000 UTC m=+858.578711624" watchObservedRunningTime="2026-09-16 14:18:48.091751542 +0000 UTC m=+858.579761679"
Sep 16 14:18:48.190896 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:48.190868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:49.148160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.148118    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"]
Sep 16 14:18:49.155367 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.155332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.158819 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.158791    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"kube-root-ca.crt\""
Sep 16 14:18:49.159510 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.159486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"default-dockercfg-f8lkf\""
Sep 16 14:18:49.159621 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.159497    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"simulator-w3-2ad565-kserve-self-signed-certs\""
Sep 16 14:18:49.161324 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.161300    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"openshift-service-ca.crt\""
Sep 16 14:18:49.164602 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.164574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"]
Sep 16 14:18:49.174224 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.174200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:49.180523 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.180495    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t7xt\" (UniqueName: \"kubernetes.io/projected/929e9227-acf9-4f5f-9818-a6e6db35ed14-kube-api-access-9t7xt\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.180637 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.180542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-model-cache\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.180637 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.180621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-dshm\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.180743 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.180685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-tmp-dir\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.180799 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.180782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-home\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.180861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.180838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/929e9227-acf9-4f5f-9818-a6e6db35ed14-tls-certs\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.281624 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.281593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-tmp-dir\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.281804 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.281658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-home\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.281804 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.281696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/929e9227-acf9-4f5f-9818-a6e6db35ed14-tls-certs\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.281804 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.281736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t7xt\" (UniqueName: \"kubernetes.io/projected/929e9227-acf9-4f5f-9818-a6e6db35ed14-kube-api-access-9t7xt\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.281804 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.281754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-model-cache\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.281804 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.281781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-dshm\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.282051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.282029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-home\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.282104 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.282080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-tmp-dir\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.282231 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.282206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-model-cache\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.283863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.283844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-dshm\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.284291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.284270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/929e9227-acf9-4f5f-9818-a6e6db35ed14-tls-certs\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.290197 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.290176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t7xt\" (UniqueName: \"kubernetes.io/projected/929e9227-acf9-4f5f-9818-a6e6db35ed14-kube-api-access-9t7xt\") pod \"simulator-w3-2ad565-kserve-58ff6795fc-5zjfz\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") " pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.468116 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.468080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:49.740054 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:49.740029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"]
Sep 16 14:18:49.741921 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:18:49.741892    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod929e9227_acf9_4f5f_9818_a6e6db35ed14.slice/crio-415213b21422f52162c4404972e2fa6b170559371abcbe0a11a53dab297c0c90 WatchSource:0}: Error finding container 415213b21422f52162c4404972e2fa6b170559371abcbe0a11a53dab297c0c90: Status 404 returned error can't find the container with id 415213b21422f52162c4404972e2fa6b170559371abcbe0a11a53dab297c0c90
Sep 16 14:18:50.084925 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:50.084835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz" event={"ID":"929e9227-acf9-4f5f-9818-a6e6db35ed14","Type":"ContainerStarted","Data":"f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e"}
Sep 16 14:18:50.084925 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:50.084878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz" event={"ID":"929e9227-acf9-4f5f-9818-a6e6db35ed14","Type":"ContainerStarted","Data":"415213b21422f52162c4404972e2fa6b170559371abcbe0a11a53dab297c0c90"}
Sep 16 14:18:50.101899 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:50.101842    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz" podStartSLOduration=1.10182768 podStartE2EDuration="1.10182768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:50.101067117 +0000 UTC m=+860.589077194" watchObservedRunningTime="2026-09-16 14:18:50.10182768 +0000 UTC m=+860.589837795"
Sep 16 14:18:52.074849 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:52.074810    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:18:55.696754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:55.696715    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:18:58.999790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:58.999753    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:18:59.081268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:59.081239    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:18:59.469215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:59.469187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:59.469393 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:59.469233    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:18:59.481397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:18:59.481372    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:19:00.146067 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:00.146037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:19:02.153849 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:02.153811    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:08.486910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:08.486857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:09.969057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:09.969024    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:19:10.133594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:10.133556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:10.237343 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:10.237263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:11.136851 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:11.136816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:13.268471 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:13.268440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:19.946825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:19.946787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:20.377609 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:20.377573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:23.757965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.757879    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"]
Sep 16 14:19:23.767160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.763365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:23.767160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.766439    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"premium-w3-2ad565-kserve-self-signed-certs\""
Sep 16 14:19:23.772593 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.772565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"]
Sep 16 14:19:23.784664 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.784635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:23.905442 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.905417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2vp6\" (UniqueName: \"kubernetes.io/projected/661395fe-d382-4970-bd6f-1cadca42c6fa-kube-api-access-x2vp6\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:23.905535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.905457    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-home\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:23.905535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.905474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-tmp-dir\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:23.905607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.905577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-dshm\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:23.905644 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.905607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-model-cache\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:23.905644 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:23.905624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/661395fe-d382-4970-bd6f-1cadca42c6fa-tls-certs\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vp6\" (UniqueName: \"kubernetes.io/projected/661395fe-d382-4970-bd6f-1cadca42c6fa-kube-api-access-x2vp6\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-home\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-tmp-dir\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-dshm\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-model-cache\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/661395fe-d382-4970-bd6f-1cadca42c6fa-tls-certs\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006816 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-home\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006857 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-tmp-dir\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.006892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.006867    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-model-cache\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.008651 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.008600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-dshm\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.008880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.008862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/661395fe-d382-4970-bd6f-1cadca42c6fa-tls-certs\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.014574 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.014552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2vp6\" (UniqueName: \"kubernetes.io/projected/661395fe-d382-4970-bd6f-1cadca42c6fa-kube-api-access-x2vp6\") pod \"premium-w3-2ad565-kserve-774ccb749b-2tfwt\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") " pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.088288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.088263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:24.371586 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:24.371562    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"]
Sep 16 14:19:24.373928 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:19:24.373897    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod661395fe_d382_4970_bd6f_1cadca42c6fa.slice/crio-77276a15934537dce878711d63d30f305bc95b7f4506502db256a14f49cf897a WatchSource:0}: Error finding container 77276a15934537dce878711d63d30f305bc95b7f4506502db256a14f49cf897a: Status 404 returned error can't find the container with id 77276a15934537dce878711d63d30f305bc95b7f4506502db256a14f49cf897a
Sep 16 14:19:25.261774 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:25.261745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt" event={"ID":"661395fe-d382-4970-bd6f-1cadca42c6fa","Type":"ContainerStarted","Data":"3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826"}
Sep 16 14:19:25.262249 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:25.261787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt" event={"ID":"661395fe-d382-4970-bd6f-1cadca42c6fa","Type":"ContainerStarted","Data":"77276a15934537dce878711d63d30f305bc95b7f4506502db256a14f49cf897a"}
Sep 16 14:19:25.279604 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:25.279561    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt" podStartSLOduration=2.279545977 podStartE2EDuration="2.279545977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:19:25.278699269 +0000 UTC m=+895.766709345" watchObservedRunningTime="2026-09-16 14:19:25.279545977 +0000 UTC m=+895.767556054"
Sep 16 14:19:25.545860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:25.545780    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:25.646189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:25.646163    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:28.482976 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:28.482940    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:19:34.088978 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:34.088931    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:34.088978 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:34.088988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:34.101170 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:34.101127    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:34.316243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:34.316217    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:19:39.311550 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:39.311509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:19:46.477870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:19:46.477838    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:20:01.309742 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:20:01.309705    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:20:07.000183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:20:07.000120    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:20:11.000827 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:20:11.000790    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:20:12.371058 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:20:12.371025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:20:18.879681 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:20:18.879647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:20:40.771164 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:20:40.771120    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:21:04.350473 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:04.350381    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:21:08.000459 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:08.000427    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:21:09.621878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:09.621846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:21:11.105855 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:11.105807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:21:11.985552 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:11.985520    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8968-8f4040"]
Sep 16 14:21:11.989259 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:11.989244    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:11.992462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:11.992445    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8t2sn\""
Sep 16 14:21:12.007497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.007476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8968-8f4040"]
Sep 16 14:21:12.113708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.113687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnnl4\" (UniqueName: \"kubernetes.io/projected/ee4126a7-01eb-439c-8b81-976022033a94-kube-api-access-mnnl4\") pod \"test-curl-8968-8f4040\" (UID: \"ee4126a7-01eb-439c-8b81-976022033a94\") " pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:12.214352 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.214325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnnl4\" (UniqueName: \"kubernetes.io/projected/ee4126a7-01eb-439c-8b81-976022033a94-kube-api-access-mnnl4\") pod \"test-curl-8968-8f4040\" (UID: \"ee4126a7-01eb-439c-8b81-976022033a94\") " pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:12.224260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.224237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnnl4\" (UniqueName: \"kubernetes.io/projected/ee4126a7-01eb-439c-8b81-976022033a94-kube-api-access-mnnl4\") pod \"test-curl-8968-8f4040\" (UID: \"ee4126a7-01eb-439c-8b81-976022033a94\") " pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:12.298078 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.298011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:12.577186 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.577160    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8968-8f4040"]
Sep 16 14:21:12.578246 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:21:12.578217    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee4126a7_01eb_439c_8b81_976022033a94.slice/crio-388607f7830b56c2e6ed91d76dfa29bb922640f33fb32c5cdf9b59ffbca3eca2 WatchSource:0}: Error finding container 388607f7830b56c2e6ed91d76dfa29bb922640f33fb32c5cdf9b59ffbca3eca2: Status 404 returned error can't find the container with id 388607f7830b56c2e6ed91d76dfa29bb922640f33fb32c5cdf9b59ffbca3eca2
Sep 16 14:21:12.580206 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.580190    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:21:12.819713 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:12.819679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8968-8f4040" event={"ID":"ee4126a7-01eb-439c-8b81-976022033a94","Type":"ContainerStarted","Data":"388607f7830b56c2e6ed91d76dfa29bb922640f33fb32c5cdf9b59ffbca3eca2"}
Sep 16 14:21:16.841721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:16.841688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8968-8f4040" event={"ID":"ee4126a7-01eb-439c-8b81-976022033a94","Type":"ContainerStarted","Data":"c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3"}
Sep 16 14:21:16.860609 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:16.860546    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8968-8f4040" podStartSLOduration=2.007044956 podStartE2EDuration="5.860532513s" totalImagesPullingTime="3.853487557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:21:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:16.859295862 +0000 UTC m=+1007.347305934" watchObservedRunningTime="2026-09-16 14:21:16.860532513 +0000 UTC m=+1007.348542579"
Sep 16 14:21:17.547929 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.547900    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8968-8f4040"]
Sep 16 14:21:17.557433 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.557370    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8968-8f4040"]
Sep 16 14:21:17.683671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.683641    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8968-b8700d"]
Sep 16 14:21:17.684088 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.684074    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee4126a7-01eb-439c-8b81-976022033a94" containerName="test-curl-8968-8f4040"
Sep 16 14:21:17.684130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.684092    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee4126a7-01eb-439c-8b81-976022033a94" containerName="test-curl-8968-8f4040"
Sep 16 14:21:17.684202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.684197    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee4126a7-01eb-439c-8b81-976022033a94" containerName="test-curl-8968-8f4040"
Sep 16 14:21:17.687359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.687344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:17.698684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.698663    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8968-b8700d"]
Sep 16 14:21:17.771173 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.771132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fknd\" (UniqueName: \"kubernetes.io/projected/059e9790-43b6-4733-83ce-20e3e87a5224-kube-api-access-9fknd\") pod \"test-curl-8968-b8700d\" (UID: \"059e9790-43b6-4733-83ce-20e3e87a5224\") " pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:17.847076 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:21:17.847004    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8968-8f4040" container="test-curl-8968-8f4040"
Sep 16 14:21:17.872286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.872252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fknd\" (UniqueName: \"kubernetes.io/projected/059e9790-43b6-4733-83ce-20e3e87a5224-kube-api-access-9fknd\") pod \"test-curl-8968-b8700d\" (UID: \"059e9790-43b6-4733-83ce-20e3e87a5224\") " pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:17.882025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.881998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fknd\" (UniqueName: \"kubernetes.io/projected/059e9790-43b6-4733-83ce-20e3e87a5224-kube-api-access-9fknd\") pod \"test-curl-8968-b8700d\" (UID: \"059e9790-43b6-4733-83ce-20e3e87a5224\") " pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:17.997036 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:17.997007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:18.273442 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:18.273414    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8968-b8700d"]
Sep 16 14:21:18.276842 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:21:18.276819    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod059e9790_43b6_4733_83ce_20e3e87a5224.slice/crio-2d190f2cc28665b1574863be50c1bf8959ed1a5e8b321e787b9490b78ba0f1b8 WatchSource:0}: Error finding container 2d190f2cc28665b1574863be50c1bf8959ed1a5e8b321e787b9490b78ba0f1b8: Status 404 returned error can't find the container with id 2d190f2cc28665b1574863be50c1bf8959ed1a5e8b321e787b9490b78ba0f1b8
Sep 16 14:21:18.854519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:18.854486    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8968-b8700d" event={"ID":"059e9790-43b6-4733-83ce-20e3e87a5224","Type":"ContainerStarted","Data":"2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98"}
Sep 16 14:21:18.854885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:18.854534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8968-b8700d" event={"ID":"059e9790-43b6-4733-83ce-20e3e87a5224","Type":"ContainerStarted","Data":"2d190f2cc28665b1574863be50c1bf8959ed1a5e8b321e787b9490b78ba0f1b8"}
Sep 16 14:21:18.854885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:18.854573    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8968-8f4040" podUID="ee4126a7-01eb-439c-8b81-976022033a94" containerName="test-curl-8968-8f4040" containerID="cri-o://c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3" gracePeriod=2
Sep 16 14:21:18.878368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:18.878321    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8968-b8700d" podStartSLOduration=1.878307526 podStartE2EDuration="1.878307526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:18.877106498 +0000 UTC m=+1009.365116577" watchObservedRunningTime="2026-09-16 14:21:18.878307526 +0000 UTC m=+1009.366317592"
Sep 16 14:21:19.549650 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:19.549582    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8968-b8700d"]
Sep 16 14:21:19.560026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:19.559998    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8968-b8700d"]
Sep 16 14:21:19.859867 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:21:19.859794    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8968-b8700d" container="test-curl-8968-b8700d"
Sep 16 14:21:20.864705 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:20.864648    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8968-b8700d" podUID="059e9790-43b6-4733-83ce-20e3e87a5224" containerName="test-curl-8968-b8700d" containerID="cri-o://2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98" gracePeriod=2
Sep 16 14:21:21.072898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.072877    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:21.204518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.204462    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnnl4\" (UniqueName: \"kubernetes.io/projected/ee4126a7-01eb-439c-8b81-976022033a94-kube-api-access-mnnl4\") pod \"ee4126a7-01eb-439c-8b81-976022033a94\" (UID: \"ee4126a7-01eb-439c-8b81-976022033a94\") "
Sep 16 14:21:21.206281 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.206257    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee4126a7-01eb-439c-8b81-976022033a94-kube-api-access-mnnl4" (OuterVolumeSpecName: "kube-api-access-mnnl4") pod "ee4126a7-01eb-439c-8b81-976022033a94" (UID: "ee4126a7-01eb-439c-8b81-976022033a94"). InnerVolumeSpecName "kube-api-access-mnnl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:21:21.305582 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.305546    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnnl4\" (UniqueName: \"kubernetes.io/projected/ee4126a7-01eb-439c-8b81-976022033a94-kube-api-access-mnnl4\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:21:21.871486 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.871448    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee4126a7-01eb-439c-8b81-976022033a94" containerID="c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3" exitCode=137
Sep 16 14:21:21.871486 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.871488    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:21.871996 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.871553    2580 scope.go:117] "RemoveContainer" containerID="c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3"
Sep 16 14:21:21.880604 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.880587    2580 scope.go:117] "RemoveContainer" containerID="c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3"
Sep 16 14:21:21.880853 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:21:21.880838    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3\": container with ID starting with c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3 not found: ID does not exist" containerID="c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3"
Sep 16 14:21:21.880903 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:21.880859    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3"} err="failed to get container status \"c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3\": rpc error: code = NotFound desc = could not find container \"c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3\": container with ID starting with c89730c643ce94ae272d17b18b248e88767ea4fe5dd65966d5b4ed3cdc8905e3 not found: ID does not exist"
Sep 16 14:21:21.882471 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:21:21.882446    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8968-8f4040\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="ee4126a7-01eb-439c-8b81-976022033a94" pod="opendatahub/test-curl-8968-8f4040"
Sep 16 14:21:22.006362 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:22.006342    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee4126a7-01eb-439c-8b81-976022033a94" path="/var/lib/kubelet/pods/ee4126a7-01eb-439c-8b81-976022033a94/volumes"
Sep 16 14:21:23.051732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.051705    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:23.223385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.223357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fknd\" (UniqueName: \"kubernetes.io/projected/059e9790-43b6-4733-83ce-20e3e87a5224-kube-api-access-9fknd\") pod \"059e9790-43b6-4733-83ce-20e3e87a5224\" (UID: \"059e9790-43b6-4733-83ce-20e3e87a5224\") "
Sep 16 14:21:23.225396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.225370    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/059e9790-43b6-4733-83ce-20e3e87a5224-kube-api-access-9fknd" (OuterVolumeSpecName: "kube-api-access-9fknd") pod "059e9790-43b6-4733-83ce-20e3e87a5224" (UID: "059e9790-43b6-4733-83ce-20e3e87a5224"). InnerVolumeSpecName "kube-api-access-9fknd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:21:23.324465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.324439    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fknd\" (UniqueName: \"kubernetes.io/projected/059e9790-43b6-4733-83ce-20e3e87a5224-kube-api-access-9fknd\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:21:23.883952 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.883920    2580 generic.go:356] "Generic (PLEG): container finished" podID="059e9790-43b6-4733-83ce-20e3e87a5224" containerID="2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98" exitCode=137
Sep 16 14:21:23.884098 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.883966    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:23.884098 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.884014    2580 scope.go:117] "RemoveContainer" containerID="2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98"
Sep 16 14:21:23.893246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.893227    2580 scope.go:117] "RemoveContainer" containerID="2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98"
Sep 16 14:21:23.893496 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:21:23.893479    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98\": container with ID starting with 2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98 not found: ID does not exist" containerID="2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98"
Sep 16 14:21:23.893537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:23.893505    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98"} err="failed to get container status \"2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98\": rpc error: code = NotFound desc = could not find container \"2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98\": container with ID starting with 2d6cdced6bcb1819b658eaf0f9a8598a797b0e43569f86690081d49f3b304d98 not found: ID does not exist"
Sep 16 14:21:23.894570 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:21:23.894549    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8968-b8700d\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="059e9790-43b6-4733-83ce-20e3e87a5224" pod="opendatahub/test-curl-8968-b8700d"
Sep 16 14:21:24.005687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:24.005664    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="059e9790-43b6-4733-83ce-20e3e87a5224" path="/var/lib/kubelet/pods/059e9790-43b6-4733-83ce-20e3e87a5224/volumes"
Sep 16 14:21:34.999785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:34.999753    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:21:41.396366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:41.396333    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:21:53.887472 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:53.887439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"]
Sep 16 14:21:53.887873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:53.887861    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="059e9790-43b6-4733-83ce-20e3e87a5224" containerName="test-curl-8968-b8700d"
Sep 16 14:21:53.887873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:53.887874    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="059e9790-43b6-4733-83ce-20e3e87a5224" containerName="test-curl-8968-b8700d"
Sep 16 14:21:53.887959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:53.887951    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="059e9790-43b6-4733-83ce-20e3e87a5224" containerName="test-curl-8968-b8700d"
Sep 16 14:21:53.896210 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:53.896192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:53.899618 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:53.899596    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"distinct-2-w3-2ad565-kserve-self-signed-certs\""
Sep 16 14:21:53.912502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:53.912474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"]
Sep 16 14:21:54.020157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.020114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-tmp-dir\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.020320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.020184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2e92653-4db8-442d-b937-f84fa7b5e787-tls-certs\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.020320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.020229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-dshm\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.020320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.020290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-home\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.020431 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.020359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqwt4\" (UniqueName: \"kubernetes.io/projected/f2e92653-4db8-442d-b937-f84fa7b5e787-kube-api-access-jqwt4\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.020431 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.020389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-model-cache\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.115592 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.115564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:21:54.121694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.121674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-model-cache\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.121779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.121749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-tmp-dir\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.121833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.121787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2e92653-4db8-442d-b937-f84fa7b5e787-tls-certs\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.121954 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.121932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-dshm\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.122070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.122046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-home\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.122207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.122184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-model-cache\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.122295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.122234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-tmp-dir\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.122508 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.122482    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-home\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.122840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.122818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqwt4\" (UniqueName: \"kubernetes.io/projected/f2e92653-4db8-442d-b937-f84fa7b5e787-kube-api-access-jqwt4\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.124586 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.124559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2e92653-4db8-442d-b937-f84fa7b5e787-tls-certs\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.124586 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.124562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-dshm\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.134205 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.134180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqwt4\" (UniqueName: \"kubernetes.io/projected/f2e92653-4db8-442d-b937-f84fa7b5e787-kube-api-access-jqwt4\") pod \"distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.205250 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.205167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:21:54.493789 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:54.493758    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"]
Sep 16 14:21:54.495211 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:21:54.495187    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2e92653_4db8_442d_b937_f84fa7b5e787.slice/crio-41c681504c309e722fe770cca73ddf41c99e3efade95d263ba62e517315b1d5a WatchSource:0}: Error finding container 41c681504c309e722fe770cca73ddf41c99e3efade95d263ba62e517315b1d5a: Status 404 returned error can't find the container with id 41c681504c309e722fe770cca73ddf41c99e3efade95d263ba62e517315b1d5a
Sep 16 14:21:55.049593 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:55.049562    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz" event={"ID":"f2e92653-4db8-442d-b937-f84fa7b5e787","Type":"ContainerStarted","Data":"6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511"}
Sep 16 14:21:55.049593 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:21:55.049597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz" event={"ID":"f2e92653-4db8-442d-b937-f84fa7b5e787","Type":"ContainerStarted","Data":"41c681504c309e722fe770cca73ddf41c99e3efade95d263ba62e517315b1d5a"}
Sep 16 14:22:01.429011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.428954    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz" podStartSLOduration=8.428936876 podStartE2EDuration="8.428936876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:55.093255057 +0000 UTC m=+1045.581265133" watchObservedRunningTime="2026-09-16 14:22:01.428936876 +0000 UTC m=+1051.916946951"
Sep 16 14:22:01.429720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.429702    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"]
Sep 16 14:22:01.433411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.433392    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.437923 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.437901    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"openshift-service-ca.crt\""
Sep 16 14:22:01.438028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.437951    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"kube-root-ca.crt\""
Sep 16 14:22:01.439127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.439111    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"distinct-w0-08475e-kserve-self-signed-certs\""
Sep 16 14:22:01.439234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.439128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"default-dockercfg-bmbjz\""
Sep 16 14:22:01.453782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.453758    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"]
Sep 16 14:22:01.491526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.491495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:01.493931 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.493910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-model-cache\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.494011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.493945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-dshm\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.494057 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.494023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wkzw\" (UniqueName: \"kubernetes.io/projected/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-kube-api-access-8wkzw\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.494094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.494065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-home\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.494094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.494085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tmp-dir\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.494201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.494102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tls-certs\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkzw\" (UniqueName: \"kubernetes.io/projected/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-kube-api-access-8wkzw\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-home\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tmp-dir\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tls-certs\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-model-cache\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-dshm\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595865 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tmp-dir\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-model-cache\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.595930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.595921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-home\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.597769 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.597750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-dshm\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.598002 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.597984    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tls-certs\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.620397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.620370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wkzw\" (UniqueName: \"kubernetes.io/projected/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-kube-api-access-8wkzw\") pod \"distinct-w0-08475e-kserve-94676f9c6-tzr6t\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:01.742959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:01.742927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:02.035911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:02.035884    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"]
Sep 16 14:22:02.037728 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:22:02.037705    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc332bc40_3330_4a2d_8f2a_e7c8a40d1ff4.slice/crio-f92d88354bac338a390d716f740141de5b56c7a8d91392904af8994b1cca72fd WatchSource:0}: Error finding container f92d88354bac338a390d716f740141de5b56c7a8d91392904af8994b1cca72fd: Status 404 returned error can't find the container with id f92d88354bac338a390d716f740141de5b56c7a8d91392904af8994b1cca72fd
Sep 16 14:22:02.084283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:02.084255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t" event={"ID":"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4","Type":"ContainerStarted","Data":"f92d88354bac338a390d716f740141de5b56c7a8d91392904af8994b1cca72fd"}
Sep 16 14:22:03.091280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:03.091242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t" event={"ID":"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4","Type":"ContainerStarted","Data":"6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6"}
Sep 16 14:22:03.115627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:03.115580    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t" podStartSLOduration=2.115565112 podStartE2EDuration="2.115565112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:03.113518182 +0000 UTC m=+1053.601528258" watchObservedRunningTime="2026-09-16 14:22:03.115565112 +0000 UTC m=+1053.603575219"
Sep 16 14:22:04.206321 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:04.206287    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:22:04.206788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:04.206357    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:22:04.218599 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:04.218569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:22:05.114017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:05.113990    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:22:06.706677 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:06.706647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:06.858853 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:06.858811    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:11.743671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:11.743630    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:11.744168 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:11.743685    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:11.755646 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:11.755622    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:12.145928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:12.145854    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:22:12.401499 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:12.401426    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:22:15.409025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:15.408979    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:15.657430 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:15.657399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:16.257298 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:16.257243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:16.642441 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:16.642343    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:17.983654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:17.983614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:18.750296 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:18.750264    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:18.875699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:18.875669    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:19.607752 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:19.607721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:21.182490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:21.182462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:22.826231 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:22.826126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:24.084811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:24.084773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:24.196965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:24.196927    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:24.668179 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:24.668125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:24.846262 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:24.846227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:24.973501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:24.973470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:25.029121 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.029090    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"]
Sep 16 14:22:25.032856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.032842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.036819 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.036801    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-597ea191\""
Sep 16 14:22:25.036933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.036824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-597ea191-dockercfg-pl26q\""
Sep 16 14:22:25.044635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.044612    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"]
Sep 16 14:22:25.101568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.101548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5620ac6-5a40-4b34-8b58-5ec79e63156a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.101858 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.101639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn5vk\" (UniqueName: \"kubernetes.io/projected/c5620ac6-5a40-4b34-8b58-5ec79e63156a-kube-api-access-xn5vk\") pod \"payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.125703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.125685    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"]
Sep 16 14:22:25.129244 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.129230    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.153275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.153254    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"]
Sep 16 14:22:25.202975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.202953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec95adae-5f0d-4045-a451-0521ce531dc3-plugins-config-volume\") pod \"payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") " pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.203089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.203012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5620ac6-5a40-4b34-8b58-5ec79e63156a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.203089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.203041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx7tv\" (UniqueName: \"kubernetes.io/projected/ec95adae-5f0d-4045-a451-0521ce531dc3-kube-api-access-qx7tv\") pod \"payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") " pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.203089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.203082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn5vk\" (UniqueName: \"kubernetes.io/projected/c5620ac6-5a40-4b34-8b58-5ec79e63156a-kube-api-access-xn5vk\") pod \"payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.203823 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.203803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5620ac6-5a40-4b34-8b58-5ec79e63156a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.214528 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.214499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn5vk\" (UniqueName: \"kubernetes.io/projected/c5620ac6-5a40-4b34-8b58-5ec79e63156a-kube-api-access-xn5vk\") pod \"payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.303766 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.303707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx7tv\" (UniqueName: \"kubernetes.io/projected/ec95adae-5f0d-4045-a451-0521ce531dc3-kube-api-access-qx7tv\") pod \"payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") " pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.303856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.303786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec95adae-5f0d-4045-a451-0521ce531dc3-plugins-config-volume\") pod \"payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") " pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.304424 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.304407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec95adae-5f0d-4045-a451-0521ce531dc3-plugins-config-volume\") pod \"payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") " pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.316875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.316850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx7tv\" (UniqueName: \"kubernetes.io/projected/ec95adae-5f0d-4045-a451-0521ce531dc3-kube-api-access-qx7tv\") pod \"payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") " pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.344757 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.344736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:25.439613 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.439582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:25.454123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.454095    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:25.637309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.637282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"]
Sep 16 14:22:25.638432 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:22:25.638409    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5620ac6_5a40_4b34_8b58_5ec79e63156a.slice/crio-80e0c8aeebb029369a0d50289d0885196b3faa1191c494592f08b84c3ae7ab1b WatchSource:0}: Error finding container 80e0c8aeebb029369a0d50289d0885196b3faa1191c494592f08b84c3ae7ab1b: Status 404 returned error can't find the container with id 80e0c8aeebb029369a0d50289d0885196b3faa1191c494592f08b84c3ae7ab1b
Sep 16 14:22:25.733633 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:25.733604    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"]
Sep 16 14:22:25.734585 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:22:25.734558    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec95adae_5f0d_4045_a451_0521ce531dc3.slice/crio-82e3d91b7319d6b25705bd3b0333892a287015ebc760ca529e3b09808d50f342 WatchSource:0}: Error finding container 82e3d91b7319d6b25705bd3b0333892a287015ebc760ca529e3b09808d50f342: Status 404 returned error can't find the container with id 82e3d91b7319d6b25705bd3b0333892a287015ebc760ca529e3b09808d50f342
Sep 16 14:22:26.207677 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.207637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc" event={"ID":"ec95adae-5f0d-4045-a451-0521ce531dc3","Type":"ContainerStarted","Data":"d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66"}
Sep 16 14:22:26.208118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.207717    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc" event={"ID":"ec95adae-5f0d-4045-a451-0521ce531dc3","Type":"ContainerStarted","Data":"82e3d91b7319d6b25705bd3b0333892a287015ebc760ca529e3b09808d50f342"}
Sep 16 14:22:26.208118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.207853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:26.209258 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.209233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98" event={"ID":"c5620ac6-5a40-4b34-8b58-5ec79e63156a","Type":"ContainerStarted","Data":"752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f"}
Sep 16 14:22:26.209379 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.209275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98" event={"ID":"c5620ac6-5a40-4b34-8b58-5ec79e63156a","Type":"ContainerStarted","Data":"80e0c8aeebb029369a0d50289d0885196b3faa1191c494592f08b84c3ae7ab1b"}
Sep 16 14:22:26.209379 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.209318    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:26.245652 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.245609    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc" podStartSLOduration=1.2455958599999999 podStartE2EDuration="1.24559586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:26.243725404 +0000 UTC m=+1076.731735529" watchObservedRunningTime="2026-09-16 14:22:26.24559586 +0000 UTC m=+1076.733605929"
Sep 16 14:22:26.281539 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.281501    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98" podStartSLOduration=1.281489456 podStartE2EDuration="1.281489456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:26.278642754 +0000 UTC m=+1076.766652830" watchObservedRunningTime="2026-09-16 14:22:26.281489456 +0000 UTC m=+1076.769499534"
Sep 16 14:22:26.775288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:26.775259    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:28.000467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:28.000437    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:22:28.575950 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:28.575921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:34.908779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:34.908718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:37.217245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:37.217212    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:22:37.217644 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:37.217361    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:22:38.218349 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:38.218315    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:22:47.247670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:47.247631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:48.498532 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:48.498490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:49.375455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.375427    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"]
Sep 16 14:22:49.379363 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.379345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"
Sep 16 14:22:49.384213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.384195    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:22:49.384493 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.384469    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:22:49.411893 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.411868    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"]
Sep 16 14:22:49.428568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.428537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09ce8798-26c0-4fa6-a3fa-824b3283dc36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-597ea191-2rshk\" (UID: \"09ce8798-26c0-4fa6-a3fa-824b3283dc36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"
Sep 16 14:22:49.529054 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.529026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09ce8798-26c0-4fa6-a3fa-824b3283dc36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-597ea191-2rshk\" (UID: \"09ce8798-26c0-4fa6-a3fa-824b3283dc36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"
Sep 16 14:22:49.529605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.529586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09ce8798-26c0-4fa6-a3fa-824b3283dc36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-597ea191-2rshk\" (UID: \"09ce8798-26c0-4fa6-a3fa-824b3283dc36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"
Sep 16 14:22:49.689518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.689467    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"
Sep 16 14:22:49.979777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:49.979754    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"]
Sep 16 14:22:49.981483 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:22:49.981453    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ce8798_26c0_4fa6_a3fa_824b3283dc36.slice/crio-09a80bd4bd3921e0286890254fbdfe84ff35717e474de41407b2b38a1dc7112c WatchSource:0}: Error finding container 09a80bd4bd3921e0286890254fbdfe84ff35717e474de41407b2b38a1dc7112c: Status 404 returned error can't find the container with id 09a80bd4bd3921e0286890254fbdfe84ff35717e474de41407b2b38a1dc7112c
Sep 16 14:22:50.153935 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:50.153904    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:50.343953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:50.343878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk" event={"ID":"09ce8798-26c0-4fa6-a3fa-824b3283dc36","Type":"ContainerStarted","Data":"09a80bd4bd3921e0286890254fbdfe84ff35717e474de41407b2b38a1dc7112c"}
Sep 16 14:22:51.559108 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:51.559055    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:52.848854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:52.848822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:53.244065 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:22:53.244035    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ce8798_26c0_4fa6_a3fa_824b3283dc36.slice/crio-conmon-34a315ddb45265bd9e3a0d9a3ad99a6a64986f194f379ecd03b72cf93f39d9cd.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ce8798_26c0_4fa6_a3fa_824b3283dc36.slice/crio-34a315ddb45265bd9e3a0d9a3ad99a6a64986f194f379ecd03b72cf93f39d9cd.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:22:53.364672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:53.364644    2580 generic.go:356] "Generic (PLEG): container finished" podID="09ce8798-26c0-4fa6-a3fa-824b3283dc36" containerID="34a315ddb45265bd9e3a0d9a3ad99a6a64986f194f379ecd03b72cf93f39d9cd" exitCode=0
Sep 16 14:22:53.364780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:53.364727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk" event={"ID":"09ce8798-26c0-4fa6-a3fa-824b3283dc36","Type":"ContainerDied","Data":"34a315ddb45265bd9e3a0d9a3ad99a6a64986f194f379ecd03b72cf93f39d9cd"}
Sep 16 14:22:54.509025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:54.508997    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"
Sep 16 14:22:54.575050 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:54.575021    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09ce8798-26c0-4fa6-a3fa-824b3283dc36-maas-api-service-ca\") pod \"09ce8798-26c0-4fa6-a3fa-824b3283dc36\" (UID: \"09ce8798-26c0-4fa6-a3fa-824b3283dc36\") "
Sep 16 14:22:54.575335 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:54.575313    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ce8798-26c0-4fa6-a3fa-824b3283dc36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09ce8798-26c0-4fa6-a3fa-824b3283dc36" (UID: "09ce8798-26c0-4fa6-a3fa-824b3283dc36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:22:54.575448 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:54.575432    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09ce8798-26c0-4fa6-a3fa-824b3283dc36-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:22:54.837990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:54.837961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:22:55.376466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:55.376435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk" event={"ID":"09ce8798-26c0-4fa6-a3fa-824b3283dc36","Type":"ContainerDied","Data":"09a80bd4bd3921e0286890254fbdfe84ff35717e474de41407b2b38a1dc7112c"}
Sep 16 14:22:55.376466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:55.376463    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"
Sep 16 14:22:55.376669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:55.376486    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09a80bd4bd3921e0286890254fbdfe84ff35717e474de41407b2b38a1dc7112c"
Sep 16 14:22:55.521915 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:22:55.521894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:00.408245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:00.408205    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:01.447678 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.447646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:01.510388 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.510357    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"]
Sep 16 14:23:01.510604 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.510578    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc" podUID="ec95adae-5f0d-4045-a451-0521ce531dc3" containerName="payload-processing" containerID="cri-o://d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66" gracePeriod=30
Sep 16 14:23:01.599462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.599440    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"]
Sep 16 14:23:01.599630 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.599611    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98" podUID="c5620ac6-5a40-4b34-8b58-5ec79e63156a" containerName="payload-pre-processing" containerID="cri-o://752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f" gracePeriod=30
Sep 16 14:23:01.839294 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.839269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:23:01.857340 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.857309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:01.866971 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.866949    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:23:01.940767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.940743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx7tv\" (UniqueName: \"kubernetes.io/projected/ec95adae-5f0d-4045-a451-0521ce531dc3-kube-api-access-qx7tv\") pod \"ec95adae-5f0d-4045-a451-0521ce531dc3\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") "
Sep 16 14:23:01.940934 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.940789    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn5vk\" (UniqueName: \"kubernetes.io/projected/c5620ac6-5a40-4b34-8b58-5ec79e63156a-kube-api-access-xn5vk\") pod \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") "
Sep 16 14:23:01.940934 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.940829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5620ac6-5a40-4b34-8b58-5ec79e63156a-plugins-config-volume\") pod \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\" (UID: \"c5620ac6-5a40-4b34-8b58-5ec79e63156a\") "
Sep 16 14:23:01.940934 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.940881    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec95adae-5f0d-4045-a451-0521ce531dc3-plugins-config-volume\") pod \"ec95adae-5f0d-4045-a451-0521ce531dc3\" (UID: \"ec95adae-5f0d-4045-a451-0521ce531dc3\") "
Sep 16 14:23:01.941324 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.941292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5620ac6-5a40-4b34-8b58-5ec79e63156a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c5620ac6-5a40-4b34-8b58-5ec79e63156a" (UID: "c5620ac6-5a40-4b34-8b58-5ec79e63156a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:01.941610 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.941583    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec95adae-5f0d-4045-a451-0521ce531dc3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec95adae-5f0d-4045-a451-0521ce531dc3" (UID: "ec95adae-5f0d-4045-a451-0521ce531dc3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:01.943313 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.943285    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec95adae-5f0d-4045-a451-0521ce531dc3-kube-api-access-qx7tv" (OuterVolumeSpecName: "kube-api-access-qx7tv") pod "ec95adae-5f0d-4045-a451-0521ce531dc3" (UID: "ec95adae-5f0d-4045-a451-0521ce531dc3"). InnerVolumeSpecName "kube-api-access-qx7tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:01.943801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:01.943771    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5620ac6-5a40-4b34-8b58-5ec79e63156a-kube-api-access-xn5vk" (OuterVolumeSpecName: "kube-api-access-xn5vk") pod "c5620ac6-5a40-4b34-8b58-5ec79e63156a" (UID: "c5620ac6-5a40-4b34-8b58-5ec79e63156a"). InnerVolumeSpecName "kube-api-access-xn5vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:02.000529 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.000505    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:23:02.042148 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.042120    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx7tv\" (UniqueName: \"kubernetes.io/projected/ec95adae-5f0d-4045-a451-0521ce531dc3-kube-api-access-qx7tv\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:02.042248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.042166    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn5vk\" (UniqueName: \"kubernetes.io/projected/c5620ac6-5a40-4b34-8b58-5ec79e63156a-kube-api-access-xn5vk\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:02.042248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.042181    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5620ac6-5a40-4b34-8b58-5ec79e63156a-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:02.042248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.042194    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec95adae-5f0d-4045-a451-0521ce531dc3-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:02.165698 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.165671    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:02.227774 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.227750    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"]
Sep 16 14:23:02.231496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.231476    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-597ea191-2rshk"]
Sep 16 14:23:02.415279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.415193    2580 generic.go:356] "Generic (PLEG): container finished" podID="ec95adae-5f0d-4045-a451-0521ce531dc3" containerID="d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66" exitCode=0
Sep 16 14:23:02.415279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.415255    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"
Sep 16 14:23:02.415517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.415281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc" event={"ID":"ec95adae-5f0d-4045-a451-0521ce531dc3","Type":"ContainerDied","Data":"d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66"}
Sep 16 14:23:02.415517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.415342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc" event={"ID":"ec95adae-5f0d-4045-a451-0521ce531dc3","Type":"ContainerDied","Data":"82e3d91b7319d6b25705bd3b0333892a287015ebc760ca529e3b09808d50f342"}
Sep 16 14:23:02.415517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.415371    2580 scope.go:117] "RemoveContainer" containerID="d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66"
Sep 16 14:23:02.417095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.416819    2580 generic.go:356] "Generic (PLEG): container finished" podID="c5620ac6-5a40-4b34-8b58-5ec79e63156a" containerID="752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f" exitCode=0
Sep 16 14:23:02.417095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.416854    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98" event={"ID":"c5620ac6-5a40-4b34-8b58-5ec79e63156a","Type":"ContainerDied","Data":"752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f"}
Sep 16 14:23:02.417095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.416891    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"
Sep 16 14:23:02.417095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.416898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98" event={"ID":"c5620ac6-5a40-4b34-8b58-5ec79e63156a","Type":"ContainerDied","Data":"80e0c8aeebb029369a0d50289d0885196b3faa1191c494592f08b84c3ae7ab1b"}
Sep 16 14:23:02.424470 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.424450    2580 scope.go:117] "RemoveContainer" containerID="d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66"
Sep 16 14:23:02.424748 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:02.424732    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66\": container with ID starting with d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66 not found: ID does not exist" containerID="d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66"
Sep 16 14:23:02.424829 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.424755    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66"} err="failed to get container status \"d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66\": rpc error: code = NotFound desc = could not find container \"d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66\": container with ID starting with d0c51f8eecc692731dd5daa531bf5ad7a0f88b8ef2f018149a4b4e1b53d2dd66 not found: ID does not exist"
Sep 16 14:23:02.424829 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.424774    2580 scope.go:117] "RemoveContainer" containerID="752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f"
Sep 16 14:23:02.432088 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.432073    2580 scope.go:117] "RemoveContainer" containerID="752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f"
Sep 16 14:23:02.432395 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:02.432377    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f\": container with ID starting with 752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f not found: ID does not exist" containerID="752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f"
Sep 16 14:23:02.432471 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.432401    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f"} err="failed to get container status \"752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f\": rpc error: code = NotFound desc = could not find container \"752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f\": container with ID starting with 752084b81f5ffde354b62d81a319e869fad2bed2f1ec82f2a66337678442204f not found: ID does not exist"
Sep 16 14:23:02.450266 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.450241    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"]
Sep 16 14:23:02.464551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.464530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:02.467850 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.467831    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-597ea191-77fc5f7946-dbw98"]
Sep 16 14:23:02.549090 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.549068    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"]
Sep 16 14:23:02.551255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.551235    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-597ea191-7888fd65f-bw6lc"]
Sep 16 14:23:02.677917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:02.677832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:03.065771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:03.065747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:03.228120 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:03.228087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:03.404270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:03.404188    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:04.007077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:04.007049    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ce8798-26c0-4fa6-a3fa-824b3283dc36" path="/var/lib/kubelet/pods/09ce8798-26c0-4fa6-a3fa-824b3283dc36/volumes"
Sep 16 14:23:04.007459 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:04.007432    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5620ac6-5a40-4b34-8b58-5ec79e63156a" path="/var/lib/kubelet/pods/c5620ac6-5a40-4b34-8b58-5ec79e63156a/volumes"
Sep 16 14:23:04.007738 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:04.007726    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec95adae-5f0d-4045-a451-0521ce531dc3" path="/var/lib/kubelet/pods/ec95adae-5f0d-4045-a451-0521ce531dc3/volumes"
Sep 16 14:23:04.022520 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:04.022491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:04.381634 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:04.381554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:05.585046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.585013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:05.807656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.807631    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"]
Sep 16 14:23:05.808058 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808046    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5620ac6-5a40-4b34-8b58-5ec79e63156a" containerName="payload-pre-processing"
Sep 16 14:23:05.808110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808060    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5620ac6-5a40-4b34-8b58-5ec79e63156a" containerName="payload-pre-processing"
Sep 16 14:23:05.808110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808069    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec95adae-5f0d-4045-a451-0521ce531dc3" containerName="payload-processing"
Sep 16 14:23:05.808110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808075    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec95adae-5f0d-4045-a451-0521ce531dc3" containerName="payload-processing"
Sep 16 14:23:05.808110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808087    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ce8798-26c0-4fa6-a3fa-824b3283dc36" containerName="revoke-keys"
Sep 16 14:23:05.808110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808092    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ce8798-26c0-4fa6-a3fa-824b3283dc36" containerName="revoke-keys"
Sep 16 14:23:05.808285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808193    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5620ac6-5a40-4b34-8b58-5ec79e63156a" containerName="payload-pre-processing"
Sep 16 14:23:05.808285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808203    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ce8798-26c0-4fa6-a3fa-824b3283dc36" containerName="revoke-keys"
Sep 16 14:23:05.808285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.808210    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec95adae-5f0d-4045-a451-0521ce531dc3" containerName="payload-processing"
Sep 16 14:23:05.812749 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.812727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"
Sep 16 14:23:05.815754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.815732    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:23:05.816264 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.816248    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:23:05.823234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.823214    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"]
Sep 16 14:23:05.877413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.877391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a241be-37e6-432b-8278-22510e0137b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5\" (UID: \"91a241be-37e6-432b-8278-22510e0137b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"
Sep 16 14:23:05.978372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.978348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a241be-37e6-432b-8278-22510e0137b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5\" (UID: \"91a241be-37e6-432b-8278-22510e0137b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"
Sep 16 14:23:05.978960 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:05.978941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a241be-37e6-432b-8278-22510e0137b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5\" (UID: \"91a241be-37e6-432b-8278-22510e0137b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"
Sep 16 14:23:06.123537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:06.123482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"
Sep 16 14:23:06.280728 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:06.280687    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:06.400338 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:06.400316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"]
Sep 16 14:23:06.401728 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:06.401697    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a241be_37e6_432b_8278_22510e0137b3.slice/crio-eade8d2478250ed70011435fbb452c0a24126bab59dc7d53ec01bdb6c9a7c999 WatchSource:0}: Error finding container eade8d2478250ed70011435fbb452c0a24126bab59dc7d53ec01bdb6c9a7c999: Status 404 returned error can't find the container with id eade8d2478250ed70011435fbb452c0a24126bab59dc7d53ec01bdb6c9a7c999
Sep 16 14:23:06.439572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:06.439545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5" event={"ID":"91a241be-37e6-432b-8278-22510e0137b3","Type":"ContainerStarted","Data":"eade8d2478250ed70011435fbb452c0a24126bab59dc7d53ec01bdb6c9a7c999"}
Sep 16 14:23:06.782352 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:06.782325    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:07.188155 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:07.188107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:07.311583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:07.311552    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:07.446470 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:07.446410    2580 generic.go:356] "Generic (PLEG): container finished" podID="91a241be-37e6-432b-8278-22510e0137b3" containerID="815237b5722684975ea26f969a9921f10f2dee5f57aae9d27939b42febc7c547" exitCode=0
Sep 16 14:23:07.446571 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:07.446476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5" event={"ID":"91a241be-37e6-432b-8278-22510e0137b3","Type":"ContainerDied","Data":"815237b5722684975ea26f969a9921f10f2dee5f57aae9d27939b42febc7c547"}
Sep 16 14:23:08.214724 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:08.214688    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:08.580927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:08.580898    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"
Sep 16 14:23:08.703373 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:08.703349    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a241be-37e6-432b-8278-22510e0137b3-maas-api-service-ca\") pod \"91a241be-37e6-432b-8278-22510e0137b3\" (UID: \"91a241be-37e6-432b-8278-22510e0137b3\") "
Sep 16 14:23:08.703716 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:08.703697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91a241be-37e6-432b-8278-22510e0137b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91a241be-37e6-432b-8278-22510e0137b3" (UID: "91a241be-37e6-432b-8278-22510e0137b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:08.804700 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:08.804647    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a241be-37e6-432b-8278-22510e0137b3-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:09.041204 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:09.041127    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:09.458003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:09.457971    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"
Sep 16 14:23:09.458389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:09.457974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5" event={"ID":"91a241be-37e6-432b-8278-22510e0137b3","Type":"ContainerDied","Data":"eade8d2478250ed70011435fbb452c0a24126bab59dc7d53ec01bdb6c9a7c999"}
Sep 16 14:23:09.458389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:09.458098    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eade8d2478250ed70011435fbb452c0a24126bab59dc7d53ec01bdb6c9a7c999"
Sep 16 14:23:09.625887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:09.625854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:11.804756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:11.804721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:11.983422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:11.983387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:12.422709 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:12.422678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:13.288902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.288858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:13.460380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.460349    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:23:13.509199 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.509162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:13.592854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.592790    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"]
Sep 16 14:23:13.593275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.593262    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a241be-37e6-432b-8278-22510e0137b3" containerName="revoke-keys"
Sep 16 14:23:13.593318 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.593277    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a241be-37e6-432b-8278-22510e0137b3" containerName="revoke-keys"
Sep 16 14:23:13.593359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.593350    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="91a241be-37e6-432b-8278-22510e0137b3" containerName="revoke-keys"
Sep 16 14:23:13.596496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.596482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.601590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.601571    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e5e410b2\""
Sep 16 14:23:13.601724 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.601663    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e5e410b2-dockercfg-zpmm2\""
Sep 16 14:23:13.631183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.631165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"]
Sep 16 14:23:13.748196 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.748161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5973173d-8d8d-4e4f-9549-dd61e1e991b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.748360 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.748319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfj7f\" (UniqueName: \"kubernetes.io/projected/5973173d-8d8d-4e4f-9549-dd61e1e991b0-kube-api-access-lfj7f\") pod \"payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.795132 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.795098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"]
Sep 16 14:23:13.798805 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.798785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:13.820120 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.820097    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"]
Sep 16 14:23:13.848908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.848841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5973173d-8d8d-4e4f-9549-dd61e1e991b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.849026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.848975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfj7f\" (UniqueName: \"kubernetes.io/projected/5973173d-8d8d-4e4f-9549-dd61e1e991b0-kube-api-access-lfj7f\") pod \"payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.849481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.849462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5973173d-8d8d-4e4f-9549-dd61e1e991b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.860991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.860974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfj7f\" (UniqueName: \"kubernetes.io/projected/5973173d-8d8d-4e4f-9549-dd61e1e991b0-kube-api-access-lfj7f\") pod \"payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.906958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.906936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:13.949839 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.949814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxh59\" (UniqueName: \"kubernetes.io/projected/c486e51a-4680-483a-b8e6-91fa28978202-kube-api-access-qxh59\") pod \"payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") " pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:13.949955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:13.949878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c486e51a-4680-483a-b8e6-91fa28978202-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") " pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:14.051306 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.051260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxh59\" (UniqueName: \"kubernetes.io/projected/c486e51a-4680-483a-b8e6-91fa28978202-kube-api-access-qxh59\") pod \"payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") " pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:14.051479 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.051364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c486e51a-4680-483a-b8e6-91fa28978202-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") " pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:14.052077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.052058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c486e51a-4680-483a-b8e6-91fa28978202-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") " pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:14.066690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.066669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxh59\" (UniqueName: \"kubernetes.io/projected/c486e51a-4680-483a-b8e6-91fa28978202-kube-api-access-qxh59\") pod \"payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") " pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:14.108550 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.108491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:14.192630 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.192493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"]
Sep 16 14:23:14.195317 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:14.195291    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5973173d_8d8d_4e4f_9549_dd61e1e991b0.slice/crio-d4dda2d4740075f6e7c53f065a06a7cf1d784ef77b14fa60619c53605fac43e7 WatchSource:0}: Error finding container d4dda2d4740075f6e7c53f065a06a7cf1d784ef77b14fa60619c53605fac43e7: Status 404 returned error can't find the container with id d4dda2d4740075f6e7c53f065a06a7cf1d784ef77b14fa60619c53605fac43e7
Sep 16 14:23:14.258632 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.258606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:14.385518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.385499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"]
Sep 16 14:23:14.387426 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:14.387396    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc486e51a_4680_483a_b8e6_91fa28978202.slice/crio-e90005c6383e348fc83d25516559f6ec219bb28c2811179ace6e0ecabb2e17a6 WatchSource:0}: Error finding container e90005c6383e348fc83d25516559f6ec219bb28c2811179ace6e0ecabb2e17a6: Status 404 returned error can't find the container with id e90005c6383e348fc83d25516559f6ec219bb28c2811179ace6e0ecabb2e17a6
Sep 16 14:23:14.486369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.486345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk" event={"ID":"c486e51a-4680-483a-b8e6-91fa28978202","Type":"ContainerStarted","Data":"7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41"}
Sep 16 14:23:14.486480 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.486400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk" event={"ID":"c486e51a-4680-483a-b8e6-91fa28978202","Type":"ContainerStarted","Data":"e90005c6383e348fc83d25516559f6ec219bb28c2811179ace6e0ecabb2e17a6"}
Sep 16 14:23:14.486541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.486528    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:14.487909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.487887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98" event={"ID":"5973173d-8d8d-4e4f-9549-dd61e1e991b0","Type":"ContainerStarted","Data":"87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a"}
Sep 16 14:23:14.488018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.487923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98" event={"ID":"5973173d-8d8d-4e4f-9549-dd61e1e991b0","Type":"ContainerStarted","Data":"d4dda2d4740075f6e7c53f065a06a7cf1d784ef77b14fa60619c53605fac43e7"}
Sep 16 14:23:14.488018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.487954    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:14.521433 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.521381    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk" podStartSLOduration=1.5213668089999999 podStartE2EDuration="1.521366809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:14.521363956 +0000 UTC m=+1125.009374034" watchObservedRunningTime="2026-09-16 14:23:14.521366809 +0000 UTC m=+1125.009376878"
Sep 16 14:23:14.558624 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:14.558587    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98" podStartSLOduration=1.558573132 podStartE2EDuration="1.558573132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:14.555132958 +0000 UTC m=+1125.043143034" watchObservedRunningTime="2026-09-16 14:23:14.558573132 +0000 UTC m=+1125.046583208"
Sep 16 14:23:15.782153 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:15.782111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:16.841800 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:16.841759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:16.928509 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:16.928481    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"]
Sep 16 14:23:16.928738 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:16.928709    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz" podUID="e70d8e47-235f-4205-bf50-33b8226bb575" containerName="payload-pre-processing" containerID="cri-o://6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a" gracePeriod=30
Sep 16 14:23:17.140648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.140613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:17.172773 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.172753    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:23:17.282462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.282424    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzgbg\" (UniqueName: \"kubernetes.io/projected/e70d8e47-235f-4205-bf50-33b8226bb575-kube-api-access-kzgbg\") pod \"e70d8e47-235f-4205-bf50-33b8226bb575\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") "
Sep 16 14:23:17.282644 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.282588    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e70d8e47-235f-4205-bf50-33b8226bb575-plugins-config-volume\") pod \"e70d8e47-235f-4205-bf50-33b8226bb575\" (UID: \"e70d8e47-235f-4205-bf50-33b8226bb575\") "
Sep 16 14:23:17.283085 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.283052    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70d8e47-235f-4205-bf50-33b8226bb575-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e70d8e47-235f-4205-bf50-33b8226bb575" (UID: "e70d8e47-235f-4205-bf50-33b8226bb575"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:17.284444 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.284423    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70d8e47-235f-4205-bf50-33b8226bb575-kube-api-access-kzgbg" (OuterVolumeSpecName: "kube-api-access-kzgbg") pod "e70d8e47-235f-4205-bf50-33b8226bb575" (UID: "e70d8e47-235f-4205-bf50-33b8226bb575"). InnerVolumeSpecName "kube-api-access-kzgbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:17.292477 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.292440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:17.384210 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.384120    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzgbg\" (UniqueName: \"kubernetes.io/projected/e70d8e47-235f-4205-bf50-33b8226bb575-kube-api-access-kzgbg\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:17.384210 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.384169    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e70d8e47-235f-4205-bf50-33b8226bb575-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:17.505580 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.505551    2580 generic.go:356] "Generic (PLEG): container finished" podID="e70d8e47-235f-4205-bf50-33b8226bb575" containerID="6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a" exitCode=0
Sep 16 14:23:17.505680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.505618    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"
Sep 16 14:23:17.505680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.505641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz" event={"ID":"e70d8e47-235f-4205-bf50-33b8226bb575","Type":"ContainerDied","Data":"6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a"}
Sep 16 14:23:17.505776 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.505716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz" event={"ID":"e70d8e47-235f-4205-bf50-33b8226bb575","Type":"ContainerDied","Data":"7809be8145f9905ce8b95059f784b4fbe2a4f00c645241f77b3a6789532821c2"}
Sep 16 14:23:17.505776 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.505740    2580 scope.go:117] "RemoveContainer" containerID="6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a"
Sep 16 14:23:17.514723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.514704    2580 scope.go:117] "RemoveContainer" containerID="6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a"
Sep 16 14:23:17.515072 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:17.514984    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a\": container with ID starting with 6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a not found: ID does not exist" containerID="6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a"
Sep 16 14:23:17.515072 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.515022    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a"} err="failed to get container status \"6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a\": rpc error: code = NotFound desc = could not find container \"6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a\": container with ID starting with 6232fe2cc727e242397bfa168322f8d3776532a6c2b5c0175582c18718b8cd2a not found: ID does not exist"
Sep 16 14:23:17.517955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.517933    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"]
Sep 16 14:23:17.522534 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.522502    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:17.530440 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.530418    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e24c87-sljf5"]
Sep 16 14:23:17.553518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.553499    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"]
Sep 16 14:23:17.560410 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.560392    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e24c87-79d4c6848b-2bxgz"]
Sep 16 14:23:17.763446 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.763416    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:17.945986 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:17.945947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:18.006729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:18.006691    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91a241be-37e6-432b-8278-22510e0137b3" path="/var/lib/kubelet/pods/91a241be-37e6-432b-8278-22510e0137b3/volumes"
Sep 16 14:23:18.007166 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:18.007129    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70d8e47-235f-4205-bf50-33b8226bb575" path="/var/lib/kubelet/pods/e70d8e47-235f-4205-bf50-33b8226bb575/volumes"
Sep 16 14:23:18.775638 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:18.775601    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:19.822448 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:19.822385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:21.653880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:21.653848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:22.842821 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:22.842787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:25.390459 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:25.390428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:25.495965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:25.495943    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:25.496109 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:25.496012    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:26.452289 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.452258    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:26.666715 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.666683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:26.816843 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.816810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:26.957578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.957547    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"]
Sep 16 14:23:26.957998 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.957985    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70d8e47-235f-4205-bf50-33b8226bb575" containerName="payload-pre-processing"
Sep 16 14:23:26.958062 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.958000    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70d8e47-235f-4205-bf50-33b8226bb575" containerName="payload-pre-processing"
Sep 16 14:23:26.958099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.958091    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70d8e47-235f-4205-bf50-33b8226bb575" containerName="payload-pre-processing"
Sep 16 14:23:26.965922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.965902    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"
Sep 16 14:23:26.971282 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.971264    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:23:26.971389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.971357    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:23:26.975003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.974973    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"]
Sep 16 14:23:26.984922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.984898    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"]
Sep 16 14:23:26.985095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:26.985075    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy" containerID="cri-o://c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f" gracePeriod=30
Sep 16 14:23:27.004701 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.004674    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:27.067538 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.067482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a18662-4f40-4964-b782-564769517487-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs\" (UID: \"c9a18662-4f40-4964-b782-564769517487\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"
Sep 16 14:23:27.168369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.168332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a18662-4f40-4964-b782-564769517487-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs\" (UID: \"c9a18662-4f40-4964-b782-564769517487\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"
Sep 16 14:23:27.168924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.168905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a18662-4f40-4964-b782-564769517487-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs\" (UID: \"c9a18662-4f40-4964-b782-564769517487\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"
Sep 16 14:23:27.277101 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.277079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"
Sep 16 14:23:27.303241 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.303217    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"]
Sep 16 14:23:27.308314 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.308298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"
Sep 16 14:23:27.333348 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.332885    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"]
Sep 16 14:23:27.470859 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.470825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d355a224-7bab-4f4f-bbd2-82263879463c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm\" (UID: \"d355a224-7bab-4f4f-bbd2-82263879463c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"
Sep 16 14:23:27.569240 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.569220    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"]
Sep 16 14:23:27.571194 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:27.571167    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a18662_4f40_4964_b782_564769517487.slice/crio-15375bf8002d8addc157d3d7cb1255b6d8de441c6844927bcd34d02392a773bc WatchSource:0}: Error finding container 15375bf8002d8addc157d3d7cb1255b6d8de441c6844927bcd34d02392a773bc: Status 404 returned error can't find the container with id 15375bf8002d8addc157d3d7cb1255b6d8de441c6844927bcd34d02392a773bc
Sep 16 14:23:27.571513 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.571495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d355a224-7bab-4f4f-bbd2-82263879463c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm\" (UID: \"d355a224-7bab-4f4f-bbd2-82263879463c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"
Sep 16 14:23:27.572110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.572091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d355a224-7bab-4f4f-bbd2-82263879463c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm\" (UID: \"d355a224-7bab-4f4f-bbd2-82263879463c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"
Sep 16 14:23:27.619238 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.619216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"
Sep 16 14:23:27.909454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:27.909432    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"]
Sep 16 14:23:27.912384 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:27.912363    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd355a224_7bab_4f4f_bbd2_82263879463c.slice/crio-1c0da64d313e413a746d712218a467d7c704568228b42a0cecca5b3fc24d2fe8 WatchSource:0}: Error finding container 1c0da64d313e413a746d712218a467d7c704568228b42a0cecca5b3fc24d2fe8: Status 404 returned error can't find the container with id 1c0da64d313e413a746d712218a467d7c704568228b42a0cecca5b3fc24d2fe8
Sep 16 14:23:28.090318 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:28.090248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:28.567736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:28.567710    2580 generic.go:356] "Generic (PLEG): container finished" podID="c9a18662-4f40-4964-b782-564769517487" containerID="3680deaa8c18d6a700fff78360b91b912eaa67fc3a38526613de8aea6062d410" exitCode=0
Sep 16 14:23:28.568074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:28.567795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs" event={"ID":"c9a18662-4f40-4964-b782-564769517487","Type":"ContainerDied","Data":"3680deaa8c18d6a700fff78360b91b912eaa67fc3a38526613de8aea6062d410"}
Sep 16 14:23:28.568074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:28.567873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs" event={"ID":"c9a18662-4f40-4964-b782-564769517487","Type":"ContainerStarted","Data":"15375bf8002d8addc157d3d7cb1255b6d8de441c6844927bcd34d02392a773bc"}
Sep 16 14:23:28.569385 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:28.569366    2580 generic.go:356] "Generic (PLEG): container finished" podID="d355a224-7bab-4f4f-bbd2-82263879463c" containerID="cf530ab0c7996ee430676ea58c368c68f821a4b906efc2f5013d607f4dfc4ef4" exitCode=0
Sep 16 14:23:28.569464 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:28.569449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm" event={"ID":"d355a224-7bab-4f4f-bbd2-82263879463c","Type":"ContainerDied","Data":"cf530ab0c7996ee430676ea58c368c68f821a4b906efc2f5013d607f4dfc4ef4"}
Sep 16 14:23:28.569503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:28.569484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm" event={"ID":"d355a224-7bab-4f4f-bbd2-82263879463c","Type":"ContainerStarted","Data":"1c0da64d313e413a746d712218a467d7c704568228b42a0cecca5b3fc24d2fe8"}
Sep 16 14:23:29.175082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.175051    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:29.727209 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.727189    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"
Sep 16 14:23:29.730165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.730131    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"
Sep 16 14:23:29.893774 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.893708    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a18662-4f40-4964-b782-564769517487-maas-api-service-ca\") pod \"c9a18662-4f40-4964-b782-564769517487\" (UID: \"c9a18662-4f40-4964-b782-564769517487\") "
Sep 16 14:23:29.893909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.893820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d355a224-7bab-4f4f-bbd2-82263879463c-maas-api-service-ca\") pod \"d355a224-7bab-4f4f-bbd2-82263879463c\" (UID: \"d355a224-7bab-4f4f-bbd2-82263879463c\") "
Sep 16 14:23:29.894051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.894029    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9a18662-4f40-4964-b782-564769517487-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c9a18662-4f40-4964-b782-564769517487" (UID: "c9a18662-4f40-4964-b782-564769517487"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:29.894108 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.894087    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d355a224-7bab-4f4f-bbd2-82263879463c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d355a224-7bab-4f4f-bbd2-82263879463c" (UID: "d355a224-7bab-4f4f-bbd2-82263879463c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:29.994550 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.994529    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a18662-4f40-4964-b782-564769517487-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:29.994550 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:29.994549    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d355a224-7bab-4f4f-bbd2-82263879463c-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:30.289800 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:30.289773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:30.581856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:30.581781    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"
Sep 16 14:23:30.581856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:30.581786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs" event={"ID":"c9a18662-4f40-4964-b782-564769517487","Type":"ContainerDied","Data":"15375bf8002d8addc157d3d7cb1255b6d8de441c6844927bcd34d02392a773bc"}
Sep 16 14:23:30.581856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:30.581838    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15375bf8002d8addc157d3d7cb1255b6d8de441c6844927bcd34d02392a773bc"
Sep 16 14:23:30.583197 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:30.583177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm" event={"ID":"d355a224-7bab-4f4f-bbd2-82263879463c","Type":"ContainerDied","Data":"1c0da64d313e413a746d712218a467d7c704568228b42a0cecca5b3fc24d2fe8"}
Sep 16 14:23:30.583305 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:30.583204    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"
Sep 16 14:23:30.583305 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:30.583211    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c0da64d313e413a746d712218a467d7c704568228b42a0cecca5b3fc24d2fe8"
Sep 16 14:23:31.906176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:31.906128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:32.244076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.244053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:23:32.415475 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415448    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx2hb\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-kube-api-access-tx2hb\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415602 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72f083bc-289b-4da9-92e6-cde78275a52f-istiod-ca-cert\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415602 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-envoy\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415602 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415575    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-data\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415608    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-socket\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-credential-socket\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72f083bc-289b-4da9-92e6-cde78275a52f-istio-podinfo\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-certs\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.415985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415887    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-data" (OuterVolumeSpecName: "istio-data") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:23:32.415985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415975    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:23:32.416100 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415984    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-istio-token\") pod \"72f083bc-289b-4da9-92e6-cde78275a52f\" (UID: \"72f083bc-289b-4da9-92e6-cde78275a52f\") "
Sep 16 14:23:32.416100 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.415992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:23:32.416100 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.416038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f083bc-289b-4da9-92e6-cde78275a52f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:32.416299 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.416104    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:23:32.416434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.416410    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-credential-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.416502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.416438    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.416502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.416454    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72f083bc-289b-4da9-92e6-cde78275a52f-istiod-ca-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.416502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.416468    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-data\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.416502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.416480    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-workload-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.417709 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.417678    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-kube-api-access-tx2hb" (OuterVolumeSpecName: "kube-api-access-tx2hb") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "kube-api-access-tx2hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:32.417824 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.417770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:23:32.417873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.417829    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-istio-token" (OuterVolumeSpecName: "istio-token") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:32.418073 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.418055    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72f083bc-289b-4da9-92e6-cde78275a52f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72f083bc-289b-4da9-92e6-cde78275a52f" (UID: "72f083bc-289b-4da9-92e6-cde78275a52f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:23:32.517217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.517157    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-istio-token\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.517217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.517176    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx2hb\" (UniqueName: \"kubernetes.io/projected/72f083bc-289b-4da9-92e6-cde78275a52f-kube-api-access-tx2hb\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.517217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.517185    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72f083bc-289b-4da9-92e6-cde78275a52f-istio-envoy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.517217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.517199    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72f083bc-289b-4da9-92e6-cde78275a52f-istio-podinfo\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:32.595412 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.595386    2580 generic.go:356] "Generic (PLEG): container finished" podID="72f083bc-289b-4da9-92e6-cde78275a52f" containerID="c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f" exitCode=0
Sep 16 14:23:32.595504 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.595439    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"
Sep 16 14:23:32.595547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.595439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" event={"ID":"72f083bc-289b-4da9-92e6-cde78275a52f","Type":"ContainerDied","Data":"c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f"}
Sep 16 14:23:32.595583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.595558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm" event={"ID":"72f083bc-289b-4da9-92e6-cde78275a52f","Type":"ContainerDied","Data":"73bf708bc724d943ff64366a5666fe1f6ae9f5d8faec2e96bcffcbaf7f9ef0a1"}
Sep 16 14:23:32.595618 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.595587    2580 scope.go:117] "RemoveContainer" containerID="c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f"
Sep 16 14:23:32.611794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.611779    2580 scope.go:117] "RemoveContainer" containerID="e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53"
Sep 16 14:23:32.619958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.619934    2580 scope.go:117] "RemoveContainer" containerID="c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f"
Sep 16 14:23:32.620238 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:32.620221    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f\": container with ID starting with c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f not found: ID does not exist" containerID="c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f"
Sep 16 14:23:32.620330 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.620249    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f"} err="failed to get container status \"c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f\": rpc error: code = NotFound desc = could not find container \"c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f\": container with ID starting with c6e1780574ae20bbb16545d0c4497610d03ed3c2798f6abce408fdb0f816dc8f not found: ID does not exist"
Sep 16 14:23:32.620330 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.620272    2580 scope.go:117] "RemoveContainer" containerID="e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53"
Sep 16 14:23:32.620534 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:32.620518    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53\": container with ID starting with e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53 not found: ID does not exist" containerID="e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53"
Sep 16 14:23:32.620574 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.620538    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53"} err="failed to get container status \"e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53\": rpc error: code = NotFound desc = could not find container \"e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53\": container with ID starting with e2e4ffa5a681ca8e8252bf62bac3a57cad2e664f7dbeb407bba04b2b36a88c53 not found: ID does not exist"
Sep 16 14:23:32.641648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.641629    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"]
Sep 16 14:23:32.643670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:32.643652    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e24c87-openshift-default-6c6878ddcb-dw7rm"]
Sep 16 14:23:33.348754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:33.348716    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:34.007777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:34.007737    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" path="/var/lib/kubelet/pods/72f083bc-289b-4da9-92e6-cde78275a52f/volumes"
Sep 16 14:23:34.312613 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:34.312538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:35.516952 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:35.516908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:36.753975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:36.753929    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:37.718265 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.718234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:37.749873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.749846    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"]
Sep 16 14:23:37.750112 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.750089    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk" podUID="c486e51a-4680-483a-b8e6-91fa28978202" containerName="payload-processing" containerID="cri-o://7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41" gracePeriod=30
Sep 16 14:23:37.756002 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.755967    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"]
Sep 16 14:23:37.756372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.756199    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98" podUID="5973173d-8d8d-4e4f-9549-dd61e1e991b0" containerName="payload-pre-processing" containerID="cri-o://87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a" gracePeriod=30
Sep 16 14:23:37.842590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.842544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:37.988824 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.988797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:37.998840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:37.998816    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:38.026307 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.026284    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"]
Sep 16 14:23:38.026513 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.026482    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv" podUID="32f3ec55-6797-4167-aa9f-c4b7748c36dd" containerName="payload-processing" containerID="cri-o://b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878" gracePeriod=30
Sep 16 14:23:38.064217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.064192    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxh59\" (UniqueName: \"kubernetes.io/projected/c486e51a-4680-483a-b8e6-91fa28978202-kube-api-access-qxh59\") pod \"c486e51a-4680-483a-b8e6-91fa28978202\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") "
Sep 16 14:23:38.064358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.064241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c486e51a-4680-483a-b8e6-91fa28978202-plugins-config-volume\") pod \"c486e51a-4680-483a-b8e6-91fa28978202\" (UID: \"c486e51a-4680-483a-b8e6-91fa28978202\") "
Sep 16 14:23:38.064599 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.064577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c486e51a-4680-483a-b8e6-91fa28978202-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c486e51a-4680-483a-b8e6-91fa28978202" (UID: "c486e51a-4680-483a-b8e6-91fa28978202"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:38.066178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.066151    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c486e51a-4680-483a-b8e6-91fa28978202-kube-api-access-qxh59" (OuterVolumeSpecName: "kube-api-access-qxh59") pod "c486e51a-4680-483a-b8e6-91fa28978202" (UID: "c486e51a-4680-483a-b8e6-91fa28978202"). InnerVolumeSpecName "kube-api-access-qxh59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:38.086222 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.086200    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:38.100248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.100223    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"]
Sep 16 14:23:38.107423 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.107404    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e5e410b2-5vlgs"]
Sep 16 14:23:38.146332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.146306    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:38.165288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.165263    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5973173d-8d8d-4e4f-9549-dd61e1e991b0-plugins-config-volume\") pod \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") "
Sep 16 14:23:38.165425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.165410    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfj7f\" (UniqueName: \"kubernetes.io/projected/5973173d-8d8d-4e4f-9549-dd61e1e991b0-kube-api-access-lfj7f\") pod \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\" (UID: \"5973173d-8d8d-4e4f-9549-dd61e1e991b0\") "
Sep 16 14:23:38.165587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.165566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5973173d-8d8d-4e4f-9549-dd61e1e991b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5973173d-8d8d-4e4f-9549-dd61e1e991b0" (UID: "5973173d-8d8d-4e4f-9549-dd61e1e991b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:38.165751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.165737    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5973173d-8d8d-4e4f-9549-dd61e1e991b0-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.165811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.165754    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxh59\" (UniqueName: \"kubernetes.io/projected/c486e51a-4680-483a-b8e6-91fa28978202-kube-api-access-qxh59\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.165811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.165762    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c486e51a-4680-483a-b8e6-91fa28978202-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.167524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.167495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5973173d-8d8d-4e4f-9549-dd61e1e991b0-kube-api-access-lfj7f" (OuterVolumeSpecName: "kube-api-access-lfj7f") pod "5973173d-8d8d-4e4f-9549-dd61e1e991b0" (UID: "5973173d-8d8d-4e4f-9549-dd61e1e991b0"). InnerVolumeSpecName "kube-api-access-lfj7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:38.266518 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.266436    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfj7f\" (UniqueName: \"kubernetes.io/projected/5973173d-8d8d-4e4f-9549-dd61e1e991b0-kube-api-access-lfj7f\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.294628 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.294602    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:23:38.302728 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.302706    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:38.367087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.367065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfq62\" (UniqueName: \"kubernetes.io/projected/32f3ec55-6797-4167-aa9f-c4b7748c36dd-kube-api-access-dfq62\") pod \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") "
Sep 16 14:23:38.367218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.367191    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32f3ec55-6797-4167-aa9f-c4b7748c36dd-plugins-config-volume\") pod \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\" (UID: \"32f3ec55-6797-4167-aa9f-c4b7748c36dd\") "
Sep 16 14:23:38.367516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.367496    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32f3ec55-6797-4167-aa9f-c4b7748c36dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32f3ec55-6797-4167-aa9f-c4b7748c36dd" (UID: "32f3ec55-6797-4167-aa9f-c4b7748c36dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:38.369013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.368981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32f3ec55-6797-4167-aa9f-c4b7748c36dd-kube-api-access-dfq62" (OuterVolumeSpecName: "kube-api-access-dfq62") pod "32f3ec55-6797-4167-aa9f-c4b7748c36dd" (UID: "32f3ec55-6797-4167-aa9f-c4b7748c36dd"). InnerVolumeSpecName "kube-api-access-dfq62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:38.462608 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.462584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:38.468521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.468492    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfq62\" (UniqueName: \"kubernetes.io/projected/32f3ec55-6797-4167-aa9f-c4b7748c36dd-kube-api-access-dfq62\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.468521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.468519    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32f3ec55-6797-4167-aa9f-c4b7748c36dd-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.588628 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.588561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:38.633501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.633476    2580 generic.go:356] "Generic (PLEG): container finished" podID="c486e51a-4680-483a-b8e6-91fa28978202" containerID="7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41" exitCode=0
Sep 16 14:23:38.633609 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.633543    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"
Sep 16 14:23:38.633609 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.633561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk" event={"ID":"c486e51a-4680-483a-b8e6-91fa28978202","Type":"ContainerDied","Data":"7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41"}
Sep 16 14:23:38.633710 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.633658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk" event={"ID":"c486e51a-4680-483a-b8e6-91fa28978202","Type":"ContainerDied","Data":"e90005c6383e348fc83d25516559f6ec219bb28c2811179ace6e0ecabb2e17a6"}
Sep 16 14:23:38.633710 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.633696    2580 scope.go:117] "RemoveContainer" containerID="7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41"
Sep 16 14:23:38.634947 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.634924    2580 generic.go:356] "Generic (PLEG): container finished" podID="32f3ec55-6797-4167-aa9f-c4b7748c36dd" containerID="b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878" exitCode=0
Sep 16 14:23:38.635034 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.634975    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"
Sep 16 14:23:38.635034 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.635000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv" event={"ID":"32f3ec55-6797-4167-aa9f-c4b7748c36dd","Type":"ContainerDied","Data":"b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878"}
Sep 16 14:23:38.635169 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.635057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv" event={"ID":"32f3ec55-6797-4167-aa9f-c4b7748c36dd","Type":"ContainerDied","Data":"4c7ba78989c3398901d2510a179a1b82d2daa823856c72c36c44688aca62cc82"}
Sep 16 14:23:38.636524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.636498    2580 generic.go:356] "Generic (PLEG): container finished" podID="5973173d-8d8d-4e4f-9549-dd61e1e991b0" containerID="87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a" exitCode=0
Sep 16 14:23:38.636641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.636537    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"
Sep 16 14:23:38.636641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.636534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98" event={"ID":"5973173d-8d8d-4e4f-9549-dd61e1e991b0","Type":"ContainerDied","Data":"87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a"}
Sep 16 14:23:38.636756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.636649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98" event={"ID":"5973173d-8d8d-4e4f-9549-dd61e1e991b0","Type":"ContainerDied","Data":"d4dda2d4740075f6e7c53f065a06a7cf1d784ef77b14fa60619c53605fac43e7"}
Sep 16 14:23:38.644500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.644484    2580 scope.go:117] "RemoveContainer" containerID="7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41"
Sep 16 14:23:38.644739 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:38.644720    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41\": container with ID starting with 7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41 not found: ID does not exist" containerID="7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41"
Sep 16 14:23:38.644812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.644750    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41"} err="failed to get container status \"7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41\": rpc error: code = NotFound desc = could not find container \"7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41\": container with ID starting with 7781b88bd148c131a40ae92a60b0b329211566faf67f6fa37b88d7471793ec41 not found: ID does not exist"
Sep 16 14:23:38.644812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.644773    2580 scope.go:117] "RemoveContainer" containerID="b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878"
Sep 16 14:23:38.652930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.652916    2580 scope.go:117] "RemoveContainer" containerID="b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878"
Sep 16 14:23:38.653172 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:38.653149    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878\": container with ID starting with b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878 not found: ID does not exist" containerID="b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878"
Sep 16 14:23:38.653243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.653179    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878"} err="failed to get container status \"b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878\": rpc error: code = NotFound desc = could not find container \"b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878\": container with ID starting with b1786f3676c9ba6562b2cafca3521c9debc70a07724d6a3cd776ca718f161878 not found: ID does not exist"
Sep 16 14:23:38.653243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.653202    2580 scope.go:117] "RemoveContainer" containerID="87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a"
Sep 16 14:23:38.658414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.658395    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"]
Sep 16 14:23:38.660867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.660841    2580 scope.go:117] "RemoveContainer" containerID="87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a"
Sep 16 14:23:38.661096 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:23:38.661080    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a\": container with ID starting with 87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a not found: ID does not exist" containerID="87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a"
Sep 16 14:23:38.661173 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.661102    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a"} err="failed to get container status \"87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a\": rpc error: code = NotFound desc = could not find container \"87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a\": container with ID starting with 87fa3de208f1b6959c9205c0182f5c34caee10a113937b1539cd1f23a8e13b5a not found: ID does not exist"
Sep 16 14:23:38.666557 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.666537    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-46e82d-8b6qm"]
Sep 16 14:23:38.676286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.676268    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"]
Sep 16 14:23:38.679713 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.679695    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e5e410b2-5c8db8ff98-tzk98"]
Sep 16 14:23:38.692649 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.692626    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:38.705617 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.705596    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"]
Sep 16 14:23:38.715497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.715477    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e5e410b2-f844447c8-9nbnk"]
Sep 16 14:23:38.744185 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.744165    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"]
Sep 16 14:23:38.751669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.751651    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46e82d-bd8b65d9-66qnv"]
Sep 16 14:23:38.903893 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:38.903838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:39.132493 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:39.132464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:39.263029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:39.263003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:40.006624 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:40.006591    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32f3ec55-6797-4167-aa9f-c4b7748c36dd" path="/var/lib/kubelet/pods/32f3ec55-6797-4167-aa9f-c4b7748c36dd/volumes"
Sep 16 14:23:40.006966 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:40.006929    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5973173d-8d8d-4e4f-9549-dd61e1e991b0" path="/var/lib/kubelet/pods/5973173d-8d8d-4e4f-9549-dd61e1e991b0/volumes"
Sep 16 14:23:40.007260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:40.007246    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c486e51a-4680-483a-b8e6-91fa28978202" path="/var/lib/kubelet/pods/c486e51a-4680-483a-b8e6-91fa28978202/volumes"
Sep 16 14:23:40.007535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:40.007524    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9a18662-4f40-4964-b782-564769517487" path="/var/lib/kubelet/pods/c9a18662-4f40-4964-b782-564769517487/volumes"
Sep 16 14:23:40.008094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:40.008080    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d355a224-7bab-4f4f-bbd2-82263879463c" path="/var/lib/kubelet/pods/d355a224-7bab-4f4f-bbd2-82263879463c/volumes"
Sep 16 14:23:40.130298 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:40.130269    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:40.467991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:40.467959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:42.154769 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:42.154737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:23:42.373400 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:42.373372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:42.776772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:42.776736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:43.129714 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:43.129636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:44.190981 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:44.190952    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:45.171601 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:45.171570    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:46.641248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:46.641210    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:46.723674 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:46.723638    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:46.825328 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:46.825302    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:47.440234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.440197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:47.552394 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552367    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"]
Sep 16 14:23:47.552801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552789    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy"
Sep 16 14:23:47.552845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552803    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy"
Sep 16 14:23:47.552845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552814    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy"
Sep 16 14:23:47.552845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552819    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy"
Sep 16 14:23:47.552845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552827    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5973173d-8d8d-4e4f-9549-dd61e1e991b0" containerName="payload-pre-processing"
Sep 16 14:23:47.552845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552833    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5973173d-8d8d-4e4f-9549-dd61e1e991b0" containerName="payload-pre-processing"
Sep 16 14:23:47.552845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552839    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c486e51a-4680-483a-b8e6-91fa28978202" containerName="payload-processing"
Sep 16 14:23:47.552845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552844    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c486e51a-4680-483a-b8e6-91fa28978202" containerName="payload-processing"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552852    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d355a224-7bab-4f4f-bbd2-82263879463c" containerName="revoke-keys"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552857    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d355a224-7bab-4f4f-bbd2-82263879463c" containerName="revoke-keys"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552871    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a18662-4f40-4964-b782-564769517487" containerName="revoke-keys"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552876    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a18662-4f40-4964-b782-564769517487" containerName="revoke-keys"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552890    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32f3ec55-6797-4167-aa9f-c4b7748c36dd" containerName="payload-processing"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552895    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="32f3ec55-6797-4167-aa9f-c4b7748c36dd" containerName="payload-processing"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552950    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552959    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f083bc-289b-4da9-92e6-cde78275a52f" containerName="istio-proxy"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552967    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c486e51a-4680-483a-b8e6-91fa28978202" containerName="payload-processing"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552973    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="32f3ec55-6797-4167-aa9f-c4b7748c36dd" containerName="payload-processing"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552979    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9a18662-4f40-4964-b782-564769517487" containerName="revoke-keys"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552985    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5973173d-8d8d-4e4f-9549-dd61e1e991b0" containerName="payload-pre-processing"
Sep 16 14:23:47.553083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.552990    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d355a224-7bab-4f4f-bbd2-82263879463c" containerName="revoke-keys"
Sep 16 14:23:47.557596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.557582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.561031 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.561010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-384cb8ac\""
Sep 16 14:23:47.582948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.582925    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"]
Sep 16 14:23:47.597948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.597926    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"]
Sep 16 14:23:47.601335 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.601315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:47.604680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.604658    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-384cb8ac-dockercfg-tz6mr\""
Sep 16 14:23:47.627993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.627970    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"]
Sep 16 14:23:47.635884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.635862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:47.644013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.643989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31d77b0-40aa-40dd-b95e-abd3811e63d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.644312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.644023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-httkv\" (UniqueName: \"kubernetes.io/projected/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-kube-api-access-httkv\") pod \"payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") " pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:47.644312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.644079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf25d\" (UniqueName: \"kubernetes.io/projected/b31d77b0-40aa-40dd-b95e-abd3811e63d8-kube-api-access-sf25d\") pod \"payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.644312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.644111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-plugins-config-volume\") pod \"payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") " pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:47.745419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.745392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31d77b0-40aa-40dd-b95e-abd3811e63d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.745575 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.745424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-httkv\" (UniqueName: \"kubernetes.io/projected/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-kube-api-access-httkv\") pod \"payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") " pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:47.745575 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.745463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf25d\" (UniqueName: \"kubernetes.io/projected/b31d77b0-40aa-40dd-b95e-abd3811e63d8-kube-api-access-sf25d\") pod \"payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.745575 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.745495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-plugins-config-volume\") pod \"payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") " pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:47.746156 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.746120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31d77b0-40aa-40dd-b95e-abd3811e63d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.746249 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.746164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-plugins-config-volume\") pod \"payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") " pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:47.770831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.770807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf25d\" (UniqueName: \"kubernetes.io/projected/b31d77b0-40aa-40dd-b95e-abd3811e63d8-kube-api-access-sf25d\") pod \"payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.770967 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.770948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-httkv\" (UniqueName: \"kubernetes.io/projected/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-kube-api-access-httkv\") pod \"payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") " pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:47.867841 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.867806    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:47.912570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:47.912545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:48.170077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.170051    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"]
Sep 16 14:23:48.171403 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:48.171377    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31d77b0_40aa_40dd_b95e_abd3811e63d8.slice/crio-5f2dfad93f2c3b16d9dac874c0141ed8748196a21cfdd9be8457eb5215c30cbd WatchSource:0}: Error finding container 5f2dfad93f2c3b16d9dac874c0141ed8748196a21cfdd9be8457eb5215c30cbd: Status 404 returned error can't find the container with id 5f2dfad93f2c3b16d9dac874c0141ed8748196a21cfdd9be8457eb5215c30cbd
Sep 16 14:23:48.206707 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.206686    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"]
Sep 16 14:23:48.219236 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:48.219214    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986f3561_e4d4_4f64_af04_4a3c1ac2b1d5.slice/crio-fe81999835cc8ab389076500154e47341252ba64c45d45e60c2fe396464490ca WatchSource:0}: Error finding container fe81999835cc8ab389076500154e47341252ba64c45d45e60c2fe396464490ca: Status 404 returned error can't find the container with id fe81999835cc8ab389076500154e47341252ba64c45d45e60c2fe396464490ca
Sep 16 14:23:48.236691 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.236665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:48.584745 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.584714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:48.693801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.693770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5" event={"ID":"b31d77b0-40aa-40dd-b95e-abd3811e63d8","Type":"ContainerStarted","Data":"ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6"}
Sep 16 14:23:48.694230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.693836    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:48.694230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.693860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5" event={"ID":"b31d77b0-40aa-40dd-b95e-abd3811e63d8","Type":"ContainerStarted","Data":"5f2dfad93f2c3b16d9dac874c0141ed8748196a21cfdd9be8457eb5215c30cbd"}
Sep 16 14:23:48.695380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.695360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq" event={"ID":"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5","Type":"ContainerStarted","Data":"7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412"}
Sep 16 14:23:48.695498 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.695395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq" event={"ID":"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5","Type":"ContainerStarted","Data":"fe81999835cc8ab389076500154e47341252ba64c45d45e60c2fe396464490ca"}
Sep 16 14:23:48.695498 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.695435    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:48.702183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.702158    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:48.770967 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.770922    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq" podStartSLOduration=1.770903814 podStartE2EDuration="1.770903814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:48.766938065 +0000 UTC m=+1159.254948140" watchObservedRunningTime="2026-09-16 14:23:48.770903814 +0000 UTC m=+1159.258913891"
Sep 16 14:23:48.771325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.771297    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5" podStartSLOduration=1.771287611 podStartE2EDuration="1.771287611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:48.731960235 +0000 UTC m=+1159.219970311" watchObservedRunningTime="2026-09-16 14:23:48.771287611 +0000 UTC m=+1159.259297686"
Sep 16 14:23:48.928879 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:48.928801    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:49.449953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:49.449918    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:49.579488 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:49.579442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:49.811480 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:49.811450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:50.156093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:50.156011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:51.006158 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:51.000817    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:23:52.294661 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:52.294630    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:52.878941 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:52.878910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:53.018158 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:53.018122    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:53.783210 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:53.783182    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:55.729733 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:55.729701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:56.916299 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:56.916268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:57.046306 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.046273    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:57.473355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.473323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:57.874974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.874898    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"]
Sep 16 14:23:57.878650 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.878635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"
Sep 16 14:23:57.889581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.889561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:23:57.889955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.889940    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:23:57.907809 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.907784    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"]
Sep 16 14:23:57.924422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.924389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:57.937554 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:57.937530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbd7fb9-03a5-496f-a2b8-f8507e71b276-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm\" (UID: \"9dbd7fb9-03a5-496f-a2b8-f8507e71b276\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"
Sep 16 14:23:58.038587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.038560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbd7fb9-03a5-496f-a2b8-f8507e71b276-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm\" (UID: \"9dbd7fb9-03a5-496f-a2b8-f8507e71b276\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"
Sep 16 14:23:58.039079 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.039063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbd7fb9-03a5-496f-a2b8-f8507e71b276-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm\" (UID: \"9dbd7fb9-03a5-496f-a2b8-f8507e71b276\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"
Sep 16 14:23:58.128866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.128812    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:58.189491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.189473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"
Sep 16 14:23:58.388028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.387951    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"]
Sep 16 14:23:58.391608 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.391588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.394514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.394496    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-50b57c-dockercfg-pgd9v\""
Sep 16 14:23:58.394682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.394660    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-50b57c\""
Sep 16 14:23:58.411683 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.411661    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"]
Sep 16 14:23:58.468797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.468779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"]
Sep 16 14:23:58.470692 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:58.470661    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbd7fb9_03a5_496f_a2b8_f8507e71b276.slice/crio-59f3ebe81c05c96cf59e7b8f67e20f8fc728c72994a7a4de91c6e5c56f5954ff WatchSource:0}: Error finding container 59f3ebe81c05c96cf59e7b8f67e20f8fc728c72994a7a4de91c6e5c56f5954ff: Status 404 returned error can't find the container with id 59f3ebe81c05c96cf59e7b8f67e20f8fc728c72994a7a4de91c6e5c56f5954ff
Sep 16 14:23:58.543605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.543581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b91b56-ada5-433b-8cbc-d56c84e6a775-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.543700 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.543645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dskp\" (UniqueName: \"kubernetes.io/projected/69b91b56-ada5-433b-8cbc-d56c84e6a775-kube-api-access-6dskp\") pod \"payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.644491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.644433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b91b56-ada5-433b-8cbc-d56c84e6a775-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.644592 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.644513    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dskp\" (UniqueName: \"kubernetes.io/projected/69b91b56-ada5-433b-8cbc-d56c84e6a775-kube-api-access-6dskp\") pod \"payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.645079 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.645058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b91b56-ada5-433b-8cbc-d56c84e6a775-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.655243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.655216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dskp\" (UniqueName: \"kubernetes.io/projected/69b91b56-ada5-433b-8cbc-d56c84e6a775-kube-api-access-6dskp\") pod \"payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.669481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.669458    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:23:58.701690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.701674    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:58.763651 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.763619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm" event={"ID":"9dbd7fb9-03a5-496f-a2b8-f8507e71b276","Type":"ContainerStarted","Data":"0fa7c76f1a50cb45f689c133926b8c5f75edb9a26a433ae4cdedfec300c802ac"}
Sep 16 14:23:58.763772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.763683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm" event={"ID":"9dbd7fb9-03a5-496f-a2b8-f8507e71b276","Type":"ContainerStarted","Data":"59f3ebe81c05c96cf59e7b8f67e20f8fc728c72994a7a4de91c6e5c56f5954ff"}
Sep 16 14:23:58.793682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:58.793631    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm" podStartSLOduration=1.793617089 podStartE2EDuration="1.793617089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:58.791244452 +0000 UTC m=+1169.279254530" watchObservedRunningTime="2026-09-16 14:23:58.793617089 +0000 UTC m=+1169.281627164"
Sep 16 14:23:59.006974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.006954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"]
Sep 16 14:23:59.007559 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:23:59.007539    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b91b56_ada5_433b_8cbc_d56c84e6a775.slice/crio-bfbcdee10615179b3b57a0c1ded8b7cada0723352ea0127ab1d92e18afa1dacd WatchSource:0}: Error finding container bfbcdee10615179b3b57a0c1ded8b7cada0723352ea0127ab1d92e18afa1dacd: Status 404 returned error can't find the container with id bfbcdee10615179b3b57a0c1ded8b7cada0723352ea0127ab1d92e18afa1dacd
Sep 16 14:23:59.704007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.703976    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:23:59.704191 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.704114    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:23:59.773339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.773238    2580 generic.go:356] "Generic (PLEG): container finished" podID="9dbd7fb9-03a5-496f-a2b8-f8507e71b276" containerID="0fa7c76f1a50cb45f689c133926b8c5f75edb9a26a433ae4cdedfec300c802ac" exitCode=0
Sep 16 14:23:59.773339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.773276    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm" event={"ID":"9dbd7fb9-03a5-496f-a2b8-f8507e71b276","Type":"ContainerDied","Data":"0fa7c76f1a50cb45f689c133926b8c5f75edb9a26a433ae4cdedfec300c802ac"}
Sep 16 14:23:59.775304 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.775280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw" event={"ID":"69b91b56-ada5-433b-8cbc-d56c84e6a775","Type":"ContainerStarted","Data":"c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66"}
Sep 16 14:23:59.775419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.775321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw" event={"ID":"69b91b56-ada5-433b-8cbc-d56c84e6a775","Type":"ContainerStarted","Data":"bfbcdee10615179b3b57a0c1ded8b7cada0723352ea0127ab1d92e18afa1dacd"}
Sep 16 14:23:59.775419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.775403    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:23:59.823487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.823446    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw" podStartSLOduration=1.823432415 podStartE2EDuration="1.823432415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:59.821801611 +0000 UTC m=+1170.309811687" watchObservedRunningTime="2026-09-16 14:23:59.823432415 +0000 UTC m=+1170.311442491"
Sep 16 14:23:59.988366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:23:59.988344    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:00.913876 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:00.913852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"
Sep 16 14:24:01.068192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:01.068120    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbd7fb9-03a5-496f-a2b8-f8507e71b276-maas-api-service-ca\") pod \"9dbd7fb9-03a5-496f-a2b8-f8507e71b276\" (UID: \"9dbd7fb9-03a5-496f-a2b8-f8507e71b276\") "
Sep 16 14:24:01.068462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:01.068436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dbd7fb9-03a5-496f-a2b8-f8507e71b276-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9dbd7fb9-03a5-496f-a2b8-f8507e71b276" (UID: "9dbd7fb9-03a5-496f-a2b8-f8507e71b276"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:24:01.169745 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:01.169722    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbd7fb9-03a5-496f-a2b8-f8507e71b276-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:01.198653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:01.198623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:01.787932 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:01.787893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm" event={"ID":"9dbd7fb9-03a5-496f-a2b8-f8507e71b276","Type":"ContainerDied","Data":"59f3ebe81c05c96cf59e7b8f67e20f8fc728c72994a7a4de91c6e5c56f5954ff"}
Sep 16 14:24:01.788118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:01.787952    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59f3ebe81c05c96cf59e7b8f67e20f8fc728c72994a7a4de91c6e5c56f5954ff"
Sep 16 14:24:01.788118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:01.787906    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"
Sep 16 14:24:02.472862 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:02.472833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:08.027594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:08.027558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:08.968812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:08.968781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:09.013780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.013751    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"]
Sep 16 14:24:09.014016 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.013992    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq" podUID="986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" containerName="payload-processing" containerID="cri-o://7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412" gracePeriod=30
Sep 16 14:24:09.025060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.025039    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"]
Sep 16 14:24:09.025277 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.025256    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5" podUID="b31d77b0-40aa-40dd-b95e-abd3811e63d8" containerName="payload-pre-processing" containerID="cri-o://ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6" gracePeriod=30
Sep 16 14:24:09.072362 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.072328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:09.202368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.202332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:09.287502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.287480    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:24:09.290646 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.290622    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:24:09.326353 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.326332    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"]
Sep 16 14:24:09.343751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.343728    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-384cb8ac-qlqmm"]
Sep 16 14:24:09.351271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.351243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:09.446472 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.446449    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-httkv\" (UniqueName: \"kubernetes.io/projected/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-kube-api-access-httkv\") pod \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") "
Sep 16 14:24:09.446597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.446487    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-plugins-config-volume\") pod \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\" (UID: \"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5\") "
Sep 16 14:24:09.446597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.446540    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf25d\" (UniqueName: \"kubernetes.io/projected/b31d77b0-40aa-40dd-b95e-abd3811e63d8-kube-api-access-sf25d\") pod \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") "
Sep 16 14:24:09.446597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.446570    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31d77b0-40aa-40dd-b95e-abd3811e63d8-plugins-config-volume\") pod \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\" (UID: \"b31d77b0-40aa-40dd-b95e-abd3811e63d8\") "
Sep 16 14:24:09.446902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.446876    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" (UID: "986f3561-e4d4-4f64-af04-4a3c1ac2b1d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:24:09.447010 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.446952    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b31d77b0-40aa-40dd-b95e-abd3811e63d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b31d77b0-40aa-40dd-b95e-abd3811e63d8" (UID: "b31d77b0-40aa-40dd-b95e-abd3811e63d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:24:09.448527 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.448512    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-kube-api-access-httkv" (OuterVolumeSpecName: "kube-api-access-httkv") pod "986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" (UID: "986f3561-e4d4-4f64-af04-4a3c1ac2b1d5"). InnerVolumeSpecName "kube-api-access-httkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:24:09.448696 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.448683    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31d77b0-40aa-40dd-b95e-abd3811e63d8-kube-api-access-sf25d" (OuterVolumeSpecName: "kube-api-access-sf25d") pod "b31d77b0-40aa-40dd-b95e-abd3811e63d8" (UID: "b31d77b0-40aa-40dd-b95e-abd3811e63d8"). InnerVolumeSpecName "kube-api-access-sf25d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:24:09.471446 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.471423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:09.547802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.547751    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-httkv\" (UniqueName: \"kubernetes.io/projected/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-kube-api-access-httkv\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:09.547802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.547772    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:09.547802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.547787    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf25d\" (UniqueName: \"kubernetes.io/projected/b31d77b0-40aa-40dd-b95e-abd3811e63d8-kube-api-access-sf25d\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:09.547802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.547796    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31d77b0-40aa-40dd-b95e-abd3811e63d8-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:09.839906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.839836    2580 generic.go:356] "Generic (PLEG): container finished" podID="b31d77b0-40aa-40dd-b95e-abd3811e63d8" containerID="ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6" exitCode=0
Sep 16 14:24:09.839906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.839881    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"
Sep 16 14:24:09.840104 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.839929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5" event={"ID":"b31d77b0-40aa-40dd-b95e-abd3811e63d8","Type":"ContainerDied","Data":"ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6"}
Sep 16 14:24:09.840104 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.839981    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5" event={"ID":"b31d77b0-40aa-40dd-b95e-abd3811e63d8","Type":"ContainerDied","Data":"5f2dfad93f2c3b16d9dac874c0141ed8748196a21cfdd9be8457eb5215c30cbd"}
Sep 16 14:24:09.840104 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.839997    2580 scope.go:117] "RemoveContainer" containerID="ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6"
Sep 16 14:24:09.841559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.841533    2580 generic.go:356] "Generic (PLEG): container finished" podID="986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" containerID="7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412" exitCode=0
Sep 16 14:24:09.841664 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.841576    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"
Sep 16 14:24:09.841664 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.841568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq" event={"ID":"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5","Type":"ContainerDied","Data":"7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412"}
Sep 16 14:24:09.841781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.841710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq" event={"ID":"986f3561-e4d4-4f64-af04-4a3c1ac2b1d5","Type":"ContainerDied","Data":"fe81999835cc8ab389076500154e47341252ba64c45d45e60c2fe396464490ca"}
Sep 16 14:24:09.850389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.850371    2580 scope.go:117] "RemoveContainer" containerID="ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6"
Sep 16 14:24:09.850749 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:24:09.850723    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6\": container with ID starting with ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6 not found: ID does not exist" containerID="ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6"
Sep 16 14:24:09.850809 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.850756    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6"} err="failed to get container status \"ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6\": rpc error: code = NotFound desc = could not find container \"ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6\": container with ID starting with ab562e71fe5087094af3f2cd822dc042956c28bb880e9624f2a5b1bb0630e8c6 not found: ID does not exist"
Sep 16 14:24:09.850809 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.850773    2580 scope.go:117] "RemoveContainer" containerID="7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412"
Sep 16 14:24:09.858866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.858849    2580 scope.go:117] "RemoveContainer" containerID="7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412"
Sep 16 14:24:09.859121 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:24:09.859102    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412\": container with ID starting with 7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412 not found: ID does not exist" containerID="7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412"
Sep 16 14:24:09.859188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.859127    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412"} err="failed to get container status \"7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412\": rpc error: code = NotFound desc = could not find container \"7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412\": container with ID starting with 7d2522425d1344065078050c961f676a91d1001a035be141dd4faf6285618412 not found: ID does not exist"
Sep 16 14:24:09.873400 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.873381    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"]
Sep 16 14:24:09.884384 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.884364    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-384cb8ac-546bffc8bb-kbkb5"]
Sep 16 14:24:09.900759 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.900737    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"]
Sep 16 14:24:09.903785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:09.903759    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-384cb8ac-fb799fc87-5gwvq"]
Sep 16 14:24:10.005531 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:10.005507    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" path="/var/lib/kubelet/pods/986f3561-e4d4-4f64-af04-4a3c1ac2b1d5/volumes"
Sep 16 14:24:10.005815 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:10.005803    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dbd7fb9-03a5-496f-a2b8-f8507e71b276" path="/var/lib/kubelet/pods/9dbd7fb9-03a5-496f-a2b8-f8507e71b276/volumes"
Sep 16 14:24:10.006105 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:10.006094    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b31d77b0-40aa-40dd-b95e-abd3811e63d8" path="/var/lib/kubelet/pods/b31d77b0-40aa-40dd-b95e-abd3811e63d8/volumes"
Sep 16 14:24:10.109378 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:10.109323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:10.624981 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:10.624947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:10.783074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:10.783050    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:24:11.078654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:11.078624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:11.224230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:11.224188    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:11.645048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:11.645021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:12.119764 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:12.119732    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:13.027045 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:13.027008    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:14.417627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:14.417592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:15.378773 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:15.378743    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:15.519614 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:15.519587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:15.929032 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:15.929004    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:17.023272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:17.023239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:17.205967 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:17.205938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:17.810186 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:17.810156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:17.933035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:17.932999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:18.036440 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.036409    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"]
Sep 16 14:24:18.036994 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.036975    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" containerName="payload-processing"
Sep 16 14:24:18.037089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.036997    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" containerName="payload-processing"
Sep 16 14:24:18.037089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.037027    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b31d77b0-40aa-40dd-b95e-abd3811e63d8" containerName="payload-pre-processing"
Sep 16 14:24:18.037089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.037039    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b31d77b0-40aa-40dd-b95e-abd3811e63d8" containerName="payload-pre-processing"
Sep 16 14:24:18.037089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.037073    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dbd7fb9-03a5-496f-a2b8-f8507e71b276" containerName="revoke-keys"
Sep 16 14:24:18.037089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.037081    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dbd7fb9-03a5-496f-a2b8-f8507e71b276" containerName="revoke-keys"
Sep 16 14:24:18.037392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.037209    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b31d77b0-40aa-40dd-b95e-abd3811e63d8" containerName="payload-pre-processing"
Sep 16 14:24:18.037392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.037226    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dbd7fb9-03a5-496f-a2b8-f8507e71b276" containerName="revoke-keys"
Sep 16 14:24:18.037392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.037236    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="986f3561-e4d4-4f64-af04-4a3c1ac2b1d5" containerName="payload-processing"
Sep 16 14:24:18.041545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.041528    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.044396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.044379    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-1f95faaa\""
Sep 16 14:24:18.044568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.044552    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-1f95faaa-dockercfg-sdrzq\""
Sep 16 14:24:18.052588 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.052570    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"]
Sep 16 14:24:18.103415 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.103364    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"]
Sep 16 14:24:18.106756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.106742    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.129207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.129177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"]
Sep 16 14:24:18.179922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.179893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:18.221180 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.221157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.221283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.221214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v86w4\" (UniqueName: \"kubernetes.io/projected/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-kube-api-access-v86w4\") pod \"payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.221283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.221256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cd83e-10a5-419d-ad89-0ede87e7e156-plugins-config-volume\") pod \"payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") " pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.221386 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.221285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whdvz\" (UniqueName: \"kubernetes.io/projected/8c9cd83e-10a5-419d-ad89-0ede87e7e156-kube-api-access-whdvz\") pod \"payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") " pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.322608 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.322580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cd83e-10a5-419d-ad89-0ede87e7e156-plugins-config-volume\") pod \"payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") " pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.322714 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.322624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whdvz\" (UniqueName: \"kubernetes.io/projected/8c9cd83e-10a5-419d-ad89-0ede87e7e156-kube-api-access-whdvz\") pod \"payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") " pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.322778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.322751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.322836 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.322810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v86w4\" (UniqueName: \"kubernetes.io/projected/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-kube-api-access-v86w4\") pod \"payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.323241 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.323222    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cd83e-10a5-419d-ad89-0ede87e7e156-plugins-config-volume\") pod \"payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") " pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.323419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.323404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.332745 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.332724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whdvz\" (UniqueName: \"kubernetes.io/projected/8c9cd83e-10a5-419d-ad89-0ede87e7e156-kube-api-access-whdvz\") pod \"payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") " pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.334651 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.334634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v86w4\" (UniqueName: \"kubernetes.io/projected/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-kube-api-access-v86w4\") pod \"payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.353534 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.353486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.415828 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.415592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.569868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.569831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:18.671405 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.671348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"]
Sep 16 14:24:18.682300 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:24:18.682276    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1835213b_fcbd_4e28_aa0c_a3087d4b76ee.slice/crio-d07fc93e3c9dcf6eb68a7aea83567a1a366924098dd3d3fcf0b9fc9d1044d498 WatchSource:0}: Error finding container d07fc93e3c9dcf6eb68a7aea83567a1a366924098dd3d3fcf0b9fc9d1044d498: Status 404 returned error can't find the container with id d07fc93e3c9dcf6eb68a7aea83567a1a366924098dd3d3fcf0b9fc9d1044d498
Sep 16 14:24:18.744062 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.744040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"]
Sep 16 14:24:18.746663 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:24:18.746638    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9cd83e_10a5_419d_ad89_0ede87e7e156.slice/crio-878c5f3829a6555658d1c52a60295c1c1ae42c6508f4dfd3c21ff189327671ef WatchSource:0}: Error finding container 878c5f3829a6555658d1c52a60295c1c1ae42c6508f4dfd3c21ff189327671ef: Status 404 returned error can't find the container with id 878c5f3829a6555658d1c52a60295c1c1ae42c6508f4dfd3c21ff189327671ef
Sep 16 14:24:18.799389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.799359    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:18.893978 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.893940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2" event={"ID":"8c9cd83e-10a5-419d-ad89-0ede87e7e156","Type":"ContainerStarted","Data":"e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a"}
Sep 16 14:24:18.894129 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.893999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2" event={"ID":"8c9cd83e-10a5-419d-ad89-0ede87e7e156","Type":"ContainerStarted","Data":"878c5f3829a6555658d1c52a60295c1c1ae42c6508f4dfd3c21ff189327671ef"}
Sep 16 14:24:18.894213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.894128    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:18.895588 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.895566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw" event={"ID":"1835213b-fcbd-4e28-aa0c-a3087d4b76ee","Type":"ContainerStarted","Data":"184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27"}
Sep 16 14:24:18.895671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.895608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:18.895671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.895624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw" event={"ID":"1835213b-fcbd-4e28-aa0c-a3087d4b76ee","Type":"ContainerStarted","Data":"d07fc93e3c9dcf6eb68a7aea83567a1a366924098dd3d3fcf0b9fc9d1044d498"}
Sep 16 14:24:18.933478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:18.933400    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2" podStartSLOduration=0.933382978 podStartE2EDuration="933.382978ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:24:18.931065894 +0000 UTC m=+1189.419075968" watchObservedRunningTime="2026-09-16 14:24:18.933382978 +0000 UTC m=+1189.421393057"
Sep 16 14:24:19.003306 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:19.003264    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw" podStartSLOduration=1.00324696 podStartE2EDuration="1.00324696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:24:19.002526717 +0000 UTC m=+1189.490536796" watchObservedRunningTime="2026-09-16 14:24:19.00324696 +0000 UTC m=+1189.491257037"
Sep 16 14:24:19.009490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:19.009467    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:19.508613 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:19.508578    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:24:20.466698 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:20.466663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:21.980283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:21.980251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:22.282156 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:22.282043    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:22.546482 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:22.546390    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:24.564000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:24.563946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:26.000316 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:26.000281    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:24:26.332980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:26.332881    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:26.433295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:26.433266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:26.747858 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:26.747818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:27.170107 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.170034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:27.292833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.292804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:27.378992 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.378967    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"]
Sep 16 14:24:27.383645 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.383629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:27.386636 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.386616    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f72402-dockercfg-ph7t8\""
Sep 16 14:24:27.387186 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.387172    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f72402\""
Sep 16 14:24:27.397204 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.397180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"]
Sep 16 14:24:27.503131 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.503108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx7dn\" (UniqueName: \"kubernetes.io/projected/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-kube-api-access-hx7dn\") pod \"payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:27.503243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.503196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:27.604196 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.604176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:27.604343 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.604274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx7dn\" (UniqueName: \"kubernetes.io/projected/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-kube-api-access-hx7dn\") pod \"payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:27.604822 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.604804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:27.616528 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.616506    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx7dn\" (UniqueName: \"kubernetes.io/projected/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-kube-api-access-hx7dn\") pod \"payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:27.695616 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:27.695597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:28.009211 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:28.009181    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"]
Sep 16 14:24:28.014985 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:24:28.014956    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44de50d6_2fd6_4e0a_bbca_fda842e39f6d.slice/crio-46a55b6ed70b33339b4402f9b2751ce1368782af580a8ae7bf090c3f1c9da01a WatchSource:0}: Error finding container 46a55b6ed70b33339b4402f9b2751ce1368782af580a8ae7bf090c3f1c9da01a: Status 404 returned error can't find the container with id 46a55b6ed70b33339b4402f9b2751ce1368782af580a8ae7bf090c3f1c9da01a
Sep 16 14:24:28.022774 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:28.022656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:28.955774 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:28.955740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4" event={"ID":"44de50d6-2fd6-4e0a-bbca-fda842e39f6d","Type":"ContainerStarted","Data":"f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a"}
Sep 16 14:24:28.956162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:28.955795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4" event={"ID":"44de50d6-2fd6-4e0a-bbca-fda842e39f6d","Type":"ContainerStarted","Data":"46a55b6ed70b33339b4402f9b2751ce1368782af580a8ae7bf090c3f1c9da01a"}
Sep 16 14:24:28.956162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:28.955816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:28.983780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:28.983732    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4" podStartSLOduration=1.983717053 podStartE2EDuration="1.983717053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:24:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:24:28.981499428 +0000 UTC m=+1199.469509505" watchObservedRunningTime="2026-09-16 14:24:28.983717053 +0000 UTC m=+1199.471727128"
Sep 16 14:24:29.442430 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:29.442402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:29.903505 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:29.903479    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:29.903666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:29.903541    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:29.986573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:29.986532    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"]
Sep 16 14:24:29.990610 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:29.990591    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"
Sep 16 14:24:29.993720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:29.993704    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:24:29.993845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:29.993702    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:24:30.012428 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.012403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"]
Sep 16 14:24:30.031481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.031450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ae8507-4979-4463-978e-ce7e6f452317-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt\" (UID: \"53ae8507-4979-4463-978e-ce7e6f452317\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"
Sep 16 14:24:30.132049 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.132028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ae8507-4979-4463-978e-ce7e6f452317-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt\" (UID: \"53ae8507-4979-4463-978e-ce7e6f452317\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"
Sep 16 14:24:30.132533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.132515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ae8507-4979-4463-978e-ce7e6f452317-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt\" (UID: \"53ae8507-4979-4463-978e-ce7e6f452317\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"
Sep 16 14:24:30.304367 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.304344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"
Sep 16 14:24:30.609225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.609063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"]
Sep 16 14:24:30.612070 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:24:30.612045    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ae8507_4979_4463_978e_ce7e6f452317.slice/crio-d6a77a6baf88b3f2c0bee18a95491db3ad99f97c87e140a6a51d0623268e8a9a WatchSource:0}: Error finding container d6a77a6baf88b3f2c0bee18a95491db3ad99f97c87e140a6a51d0623268e8a9a: Status 404 returned error can't find the container with id d6a77a6baf88b3f2c0bee18a95491db3ad99f97c87e140a6a51d0623268e8a9a
Sep 16 14:24:30.729881 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.729833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:30.970291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.970263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt" event={"ID":"53ae8507-4979-4463-978e-ce7e6f452317","Type":"ContainerStarted","Data":"60d0814c050522f499c483b2eb264346cc9db596290574d78f718003aef85e4c"}
Sep 16 14:24:30.970455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:30.970321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt" event={"ID":"53ae8507-4979-4463-978e-ce7e6f452317","Type":"ContainerStarted","Data":"d6a77a6baf88b3f2c0bee18a95491db3ad99f97c87e140a6a51d0623268e8a9a"}
Sep 16 14:24:31.977261 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:31.977225    2580 generic.go:356] "Generic (PLEG): container finished" podID="53ae8507-4979-4463-978e-ce7e6f452317" containerID="60d0814c050522f499c483b2eb264346cc9db596290574d78f718003aef85e4c" exitCode=0
Sep 16 14:24:31.977620 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:31.977314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt" event={"ID":"53ae8507-4979-4463-978e-ce7e6f452317","Type":"ContainerDied","Data":"60d0814c050522f499c483b2eb264346cc9db596290574d78f718003aef85e4c"}
Sep 16 14:24:32.134396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:32.134354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:33.109816 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.109790    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"
Sep 16 14:24:33.162942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.162913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ae8507-4979-4463-978e-ce7e6f452317-maas-api-service-ca\") pod \"53ae8507-4979-4463-978e-ce7e6f452317\" (UID: \"53ae8507-4979-4463-978e-ce7e6f452317\") "
Sep 16 14:24:33.163256 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.163228    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53ae8507-4979-4463-978e-ce7e6f452317-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53ae8507-4979-4463-978e-ce7e6f452317" (UID: "53ae8507-4979-4463-978e-ce7e6f452317"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:24:33.163392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.163330    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ae8507-4979-4463-978e-ce7e6f452317-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:33.304011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.303927    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:33.990007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.989977    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"
Sep 16 14:24:33.990268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.990019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt" event={"ID":"53ae8507-4979-4463-978e-ce7e6f452317","Type":"ContainerDied","Data":"d6a77a6baf88b3f2c0bee18a95491db3ad99f97c87e140a6a51d0623268e8a9a"}
Sep 16 14:24:33.990268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:33.990077    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6a77a6baf88b3f2c0bee18a95491db3ad99f97c87e140a6a51d0623268e8a9a"
Sep 16 14:24:34.670156 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:34.670116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:35.555503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:35.555472    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:36.914552 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:36.914510    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:37.079161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:37.079097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:38.392994 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:38.392965    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:39.965573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:39.965542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:24:40.341505 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.341338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:40.457544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.457514    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"]
Sep 16 14:24:40.457728 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.457706    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw" podUID="1835213b-fcbd-4e28-aa0c-a3087d4b76ee" containerName="payload-pre-processing" containerID="cri-o://184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27" gracePeriod=30
Sep 16 14:24:40.478472 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.478446    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"]
Sep 16 14:24:40.486985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.486846    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2" podUID="8c9cd83e-10a5-419d-ad89-0ede87e7e156" containerName="payload-processing" containerID="cri-o://e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a" gracePeriod=30
Sep 16 14:24:40.553492 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.553465    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:40.707050 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.707019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:40.760537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.760515    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:40.763411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.763391    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:40.829387 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.829366    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cd83e-10a5-419d-ad89-0ede87e7e156-plugins-config-volume\") pod \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") "
Sep 16 14:24:40.829506 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.829401    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-plugins-config-volume\") pod \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") "
Sep 16 14:24:40.829567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.829541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v86w4\" (UniqueName: \"kubernetes.io/projected/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-kube-api-access-v86w4\") pod \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\" (UID: \"1835213b-fcbd-4e28-aa0c-a3087d4b76ee\") "
Sep 16 14:24:40.829653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.829636    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whdvz\" (UniqueName: \"kubernetes.io/projected/8c9cd83e-10a5-419d-ad89-0ede87e7e156-kube-api-access-whdvz\") pod \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\" (UID: \"8c9cd83e-10a5-419d-ad89-0ede87e7e156\") "
Sep 16 14:24:40.829832 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.829803    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9cd83e-10a5-419d-ad89-0ede87e7e156-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c9cd83e-10a5-419d-ad89-0ede87e7e156" (UID: "8c9cd83e-10a5-419d-ad89-0ede87e7e156"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:24:40.829957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.829841    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1835213b-fcbd-4e28-aa0c-a3087d4b76ee" (UID: "1835213b-fcbd-4e28-aa0c-a3087d4b76ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:24:40.830020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.829997    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cd83e-10a5-419d-ad89-0ede87e7e156-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:40.830020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.830009    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:40.831596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.831578    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-kube-api-access-v86w4" (OuterVolumeSpecName: "kube-api-access-v86w4") pod "1835213b-fcbd-4e28-aa0c-a3087d4b76ee" (UID: "1835213b-fcbd-4e28-aa0c-a3087d4b76ee"). InnerVolumeSpecName "kube-api-access-v86w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:24:40.831736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.831708    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9cd83e-10a5-419d-ad89-0ede87e7e156-kube-api-access-whdvz" (OuterVolumeSpecName: "kube-api-access-whdvz") pod "8c9cd83e-10a5-419d-ad89-0ede87e7e156" (UID: "8c9cd83e-10a5-419d-ad89-0ede87e7e156"). InnerVolumeSpecName "kube-api-access-whdvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:24:40.917691 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.917639    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"]
Sep 16 14:24:40.922971 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.922953    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1f95faaa-znnjt"]
Sep 16 14:24:40.927481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.927453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:40.930610 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.930591    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v86w4\" (UniqueName: \"kubernetes.io/projected/1835213b-fcbd-4e28-aa0c-a3087d4b76ee-kube-api-access-v86w4\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:40.930685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:40.930614    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whdvz\" (UniqueName: \"kubernetes.io/projected/8c9cd83e-10a5-419d-ad89-0ede87e7e156-kube-api-access-whdvz\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:41.030402 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.030379    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c9cd83e-10a5-419d-ad89-0ede87e7e156" containerID="e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a" exitCode=0
Sep 16 14:24:41.030733 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.030419    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"
Sep 16 14:24:41.030733 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.030460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2" event={"ID":"8c9cd83e-10a5-419d-ad89-0ede87e7e156","Type":"ContainerDied","Data":"e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a"}
Sep 16 14:24:41.030733 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.030502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2" event={"ID":"8c9cd83e-10a5-419d-ad89-0ede87e7e156","Type":"ContainerDied","Data":"878c5f3829a6555658d1c52a60295c1c1ae42c6508f4dfd3c21ff189327671ef"}
Sep 16 14:24:41.030733 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.030522    2580 scope.go:117] "RemoveContainer" containerID="e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a"
Sep 16 14:24:41.031970 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.031948    2580 generic.go:356] "Generic (PLEG): container finished" podID="1835213b-fcbd-4e28-aa0c-a3087d4b76ee" containerID="184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27" exitCode=0
Sep 16 14:24:41.032070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.031991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw" event={"ID":"1835213b-fcbd-4e28-aa0c-a3087d4b76ee","Type":"ContainerDied","Data":"184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27"}
Sep 16 14:24:41.032070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.032042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw" event={"ID":"1835213b-fcbd-4e28-aa0c-a3087d4b76ee","Type":"ContainerDied","Data":"d07fc93e3c9dcf6eb68a7aea83567a1a366924098dd3d3fcf0b9fc9d1044d498"}
Sep 16 14:24:41.032070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.031997    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"
Sep 16 14:24:41.040917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.040902    2580 scope.go:117] "RemoveContainer" containerID="e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a"
Sep 16 14:24:41.041241 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:24:41.041213    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a\": container with ID starting with e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a not found: ID does not exist" containerID="e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a"
Sep 16 14:24:41.041344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.041246    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a"} err="failed to get container status \"e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a\": rpc error: code = NotFound desc = could not find container \"e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a\": container with ID starting with e54e07dde392d356b4837d0e7ad36cb74ab986163f2dea63a66e3cf2c3082f3a not found: ID does not exist"
Sep 16 14:24:41.041344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.041264    2580 scope.go:117] "RemoveContainer" containerID="184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27"
Sep 16 14:24:41.049426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.049412    2580 scope.go:117] "RemoveContainer" containerID="184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27"
Sep 16 14:24:41.049684 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:24:41.049667    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27\": container with ID starting with 184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27 not found: ID does not exist" containerID="184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27"
Sep 16 14:24:41.049719 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.049690    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27"} err="failed to get container status \"184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27\": rpc error: code = NotFound desc = could not find container \"184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27\": container with ID starting with 184e0139cf63e6511b196bef2b6ee31ac0f2fdf30c1c1ca1b36cfc3b604aff27 not found: ID does not exist"
Sep 16 14:24:41.111290 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.111253    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"]
Sep 16 14:24:41.118682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.118656    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1f95faaa-5c67b46d46-phfbw"]
Sep 16 14:24:41.168616 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.168554    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"]
Sep 16 14:24:41.197689 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.197660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:41.198381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.198361    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1f95faaa-6487547857-qbmh2"]
Sep 16 14:24:41.514778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:41.514754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:42.006674 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:42.006648    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1835213b-fcbd-4e28-aa0c-a3087d4b76ee" path="/var/lib/kubelet/pods/1835213b-fcbd-4e28-aa0c-a3087d4b76ee/volumes"
Sep 16 14:24:42.006971 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:42.006957    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53ae8507-4979-4463-978e-ce7e6f452317" path="/var/lib/kubelet/pods/53ae8507-4979-4463-978e-ce7e6f452317/volumes"
Sep 16 14:24:42.007295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:42.007282    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9cd83e-10a5-419d-ad89-0ede87e7e156" path="/var/lib/kubelet/pods/8c9cd83e-10a5-419d-ad89-0ede87e7e156/volumes"
Sep 16 14:24:42.142870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:42.142841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:42.382378 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:42.382302    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:43.846554 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:43.846517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:44.145405 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:44.145322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:45.355404 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.355371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:45.518271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518242    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"]
Sep 16 14:24:45.518702 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518689    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53ae8507-4979-4463-978e-ce7e6f452317" containerName="revoke-keys"
Sep 16 14:24:45.518748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518704    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="53ae8507-4979-4463-978e-ce7e6f452317" containerName="revoke-keys"
Sep 16 14:24:45.518748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518718    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9cd83e-10a5-419d-ad89-0ede87e7e156" containerName="payload-processing"
Sep 16 14:24:45.518748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518724    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9cd83e-10a5-419d-ad89-0ede87e7e156" containerName="payload-processing"
Sep 16 14:24:45.518748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518739    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1835213b-fcbd-4e28-aa0c-a3087d4b76ee" containerName="payload-pre-processing"
Sep 16 14:24:45.518748 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518744    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1835213b-fcbd-4e28-aa0c-a3087d4b76ee" containerName="payload-pre-processing"
Sep 16 14:24:45.518898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518814    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1835213b-fcbd-4e28-aa0c-a3087d4b76ee" containerName="payload-pre-processing"
Sep 16 14:24:45.518898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518822    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="53ae8507-4979-4463-978e-ce7e6f452317" containerName="revoke-keys"
Sep 16 14:24:45.518898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.518829    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9cd83e-10a5-419d-ad89-0ede87e7e156" containerName="payload-processing"
Sep 16 14:24:45.522889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.522873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"
Sep 16 14:24:45.548950 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.548930    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:24:45.571530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.571498    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"]
Sep 16 14:24:45.591233 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.591212    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:24:45.672679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.672613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06f8a28f-e5f9-48b6-9a04-122413c04286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb\" (UID: \"06f8a28f-e5f9-48b6-9a04-122413c04286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"
Sep 16 14:24:45.683927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.683905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:45.774026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.774003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06f8a28f-e5f9-48b6-9a04-122413c04286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb\" (UID: \"06f8a28f-e5f9-48b6-9a04-122413c04286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"
Sep 16 14:24:45.774583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.774565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06f8a28f-e5f9-48b6-9a04-122413c04286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb\" (UID: \"06f8a28f-e5f9-48b6-9a04-122413c04286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"
Sep 16 14:24:45.833333 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:45.833311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"
Sep 16 14:24:46.137283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:46.137263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"]
Sep 16 14:24:46.139110 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:24:46.139087    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f8a28f_e5f9_48b6_9a04_122413c04286.slice/crio-7860bc7548054a65be871489595271100794d1dc53ac0c6818c58c7f5e7050d0 WatchSource:0}: Error finding container 7860bc7548054a65be871489595271100794d1dc53ac0c6818c58c7f5e7050d0: Status 404 returned error can't find the container with id 7860bc7548054a65be871489595271100794d1dc53ac0c6818c58c7f5e7050d0
Sep 16 14:24:47.066694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:47.066630    2580 generic.go:356] "Generic (PLEG): container finished" podID="06f8a28f-e5f9-48b6-9a04-122413c04286" containerID="1579715c8d43a32f0a4363c9cd6739f78fe8e4717bd4772e357142d271028acb" exitCode=0
Sep 16 14:24:47.066990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:47.066721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb" event={"ID":"06f8a28f-e5f9-48b6-9a04-122413c04286","Type":"ContainerDied","Data":"1579715c8d43a32f0a4363c9cd6739f78fe8e4717bd4772e357142d271028acb"}
Sep 16 14:24:47.066990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:47.066764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb" event={"ID":"06f8a28f-e5f9-48b6-9a04-122413c04286","Type":"ContainerStarted","Data":"7860bc7548054a65be871489595271100794d1dc53ac0c6818c58c7f5e7050d0"}
Sep 16 14:24:47.456301 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:47.456213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:48.204190 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:48.204169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"
Sep 16 14:24:48.295089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:48.295062    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06f8a28f-e5f9-48b6-9a04-122413c04286-maas-api-service-ca\") pod \"06f8a28f-e5f9-48b6-9a04-122413c04286\" (UID: \"06f8a28f-e5f9-48b6-9a04-122413c04286\") "
Sep 16 14:24:48.295469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:48.295447    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f8a28f-e5f9-48b6-9a04-122413c04286-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06f8a28f-e5f9-48b6-9a04-122413c04286" (UID: "06f8a28f-e5f9-48b6-9a04-122413c04286"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:24:48.396423 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:48.396367    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06f8a28f-e5f9-48b6-9a04-122413c04286-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:24:49.088328 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:49.088296    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"
Sep 16 14:24:49.088500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:49.088317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb" event={"ID":"06f8a28f-e5f9-48b6-9a04-122413c04286","Type":"ContainerDied","Data":"7860bc7548054a65be871489595271100794d1dc53ac0c6818c58c7f5e7050d0"}
Sep 16 14:24:49.088500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:49.088403    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7860bc7548054a65be871489595271100794d1dc53ac0c6818c58c7f5e7050d0"
Sep 16 14:24:49.417970 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:49.417891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:50.261120 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:50.261089    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"]
Sep 16 14:24:50.305339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:50.305307    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"]
Sep 16 14:24:50.328526 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:50.328501    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"]
Sep 16 14:24:50.520680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:50.520608    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:50.962039 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:50.962004    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:51.202623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:51.202590    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:24:52.661931 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:52.661895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:53.792549 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:53.792522    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:53.968903 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:53.968873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:54.156535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:54.156453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:54.651927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:54.651894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:55.786567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:55.786530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:24:59.000818 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:59.000781    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:24:59.427230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:24:59.427099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:01.926005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:01.925974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:01.981800 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:01.981773    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"]
Sep 16 14:25:01.982068 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:01.982043    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp" podUID="9f98486f-75e8-4bb6-9e7d-842a485268f4" containerName="payload-processing" containerID="cri-o://eaf1418c99d2c3dc0d46e55dc15755f8b8f99a3d27e1ed420fbf843015424270" gracePeriod=30
Sep 16 14:25:02.022873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.022850    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"]
Sep 16 14:25:02.023043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.023024    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn" podUID="9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" containerName="payload-pre-processing" containerID="cri-o://9bf3e7fdd1c30c30ae72c612b368a45cfb81ac674e938c0c7533f7d6c0b5a13a" gracePeriod=30
Sep 16 14:25:02.128044 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.128009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:02.171519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.171485    2580 generic.go:356] "Generic (PLEG): container finished" podID="9f98486f-75e8-4bb6-9e7d-842a485268f4" containerID="eaf1418c99d2c3dc0d46e55dc15755f8b8f99a3d27e1ed420fbf843015424270" exitCode=0
Sep 16 14:25:02.171669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.171643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp" event={"ID":"9f98486f-75e8-4bb6-9e7d-842a485268f4","Type":"ContainerDied","Data":"eaf1418c99d2c3dc0d46e55dc15755f8b8f99a3d27e1ed420fbf843015424270"}
Sep 16 14:25:02.173438 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.173416    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" containerID="9bf3e7fdd1c30c30ae72c612b368a45cfb81ac674e938c0c7533f7d6c0b5a13a" exitCode=0
Sep 16 14:25:02.173559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.173505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn" event={"ID":"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d","Type":"ContainerDied","Data":"9bf3e7fdd1c30c30ae72c612b368a45cfb81ac674e938c0c7533f7d6c0b5a13a"}
Sep 16 14:25:02.289103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.289085    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:25:02.291756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.291739    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:25:02.325585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.325565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f98486f-75e8-4bb6-9e7d-842a485268f4-plugins-config-volume\") pod \"9f98486f-75e8-4bb6-9e7d-842a485268f4\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") "
Sep 16 14:25:02.325694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.325667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbrdg\" (UniqueName: \"kubernetes.io/projected/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-kube-api-access-nbrdg\") pod \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") "
Sep 16 14:25:02.325756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.325719    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-plugins-config-volume\") pod \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\" (UID: \"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d\") "
Sep 16 14:25:02.325820 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.325752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cddgj\" (UniqueName: \"kubernetes.io/projected/9f98486f-75e8-4bb6-9e7d-842a485268f4-kube-api-access-cddgj\") pod \"9f98486f-75e8-4bb6-9e7d-842a485268f4\" (UID: \"9f98486f-75e8-4bb6-9e7d-842a485268f4\") "
Sep 16 14:25:02.326061 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.326014    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f98486f-75e8-4bb6-9e7d-842a485268f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f98486f-75e8-4bb6-9e7d-842a485268f4" (UID: "9f98486f-75e8-4bb6-9e7d-842a485268f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:25:02.326180 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.326156    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f98486f-75e8-4bb6-9e7d-842a485268f4-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.326180 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.326152    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" (UID: "9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:25:02.328922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.328850    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-kube-api-access-nbrdg" (OuterVolumeSpecName: "kube-api-access-nbrdg") pod "9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" (UID: "9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d"). InnerVolumeSpecName "kube-api-access-nbrdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:02.328922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.328871    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f98486f-75e8-4bb6-9e7d-842a485268f4-kube-api-access-cddgj" (OuterVolumeSpecName: "kube-api-access-cddgj") pod "9f98486f-75e8-4bb6-9e7d-842a485268f4" (UID: "9f98486f-75e8-4bb6-9e7d-842a485268f4"). InnerVolumeSpecName "kube-api-access-cddgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:02.426916 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.426884    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.426916 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.426910    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cddgj\" (UniqueName: \"kubernetes.io/projected/9f98486f-75e8-4bb6-9e7d-842a485268f4-kube-api-access-cddgj\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.427063 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.426923    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbrdg\" (UniqueName: \"kubernetes.io/projected/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d-kube-api-access-nbrdg\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.464448 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.464422    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:02.524011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.523986    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"]
Sep 16 14:25:02.532377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.532355    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-2ad565-sqrzb"]
Sep 16 14:25:02.580232 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.580214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:02.987319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:02.987288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:03.146325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.146302    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:03.180344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.180323    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"
Sep 16 14:25:03.180443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.180364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp" event={"ID":"9f98486f-75e8-4bb6-9e7d-842a485268f4","Type":"ContainerDied","Data":"70d8ddb86e4ba2af5704686c42cfa4610fa72a0f3de01b8f99222856718893a0"}
Sep 16 14:25:03.180443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.180426    2580 scope.go:117] "RemoveContainer" containerID="eaf1418c99d2c3dc0d46e55dc15755f8b8f99a3d27e1ed420fbf843015424270"
Sep 16 14:25:03.182069 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.182047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn" event={"ID":"9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d","Type":"ContainerDied","Data":"6acad29cb4932d7f8e270097e9669087d2a2d41c06b2063f05fa523a8f5ecd04"}
Sep 16 14:25:03.182069 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.182060    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"
Sep 16 14:25:03.190419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.190399    2580 scope.go:117] "RemoveContainer" containerID="9bf3e7fdd1c30c30ae72c612b368a45cfb81ac674e938c0c7533f7d6c0b5a13a"
Sep 16 14:25:03.213990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.213967    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"]
Sep 16 14:25:03.223156 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.223116    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2ad565-86c87ccdc5-gw8vp"]
Sep 16 14:25:03.264096 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.264048    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"]
Sep 16 14:25:03.269100 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:03.269081    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ad565-79749d494-ksmpn"]
Sep 16 14:25:04.007274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:04.007231    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06f8a28f-e5f9-48b6-9a04-122413c04286" path="/var/lib/kubelet/pods/06f8a28f-e5f9-48b6-9a04-122413c04286/volumes"
Sep 16 14:25:04.007787 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:04.007745    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" path="/var/lib/kubelet/pods/9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d/volumes"
Sep 16 14:25:04.008185 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:04.008163    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f98486f-75e8-4bb6-9e7d-842a485268f4" path="/var/lib/kubelet/pods/9f98486f-75e8-4bb6-9e7d-842a485268f4/volumes"
Sep 16 14:25:04.863541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:04.863501    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:05.278409 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.278373    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz" podUID="f2e92653-4db8-442d-b937-f84fa7b5e787" containerName="main" containerID="cri-o://6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511" gracePeriod=45
Sep 16 14:25:05.322739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.322711    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt" podUID="661395fe-d382-4970-bd6f-1cadca42c6fa" containerName="main" containerID="cri-o://3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826" gracePeriod=45
Sep 16 14:25:05.345327 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.345302    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz" podUID="929e9227-acf9-4f5f-9818-a6e6db35ed14" containerName="main" containerID="cri-o://f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e" gracePeriod=45
Sep 16 14:25:05.623754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.623720    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:25:05.630889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.630868    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:25:05.633576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.633550    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:25:05.659252 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659228    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqwt4\" (UniqueName: \"kubernetes.io/projected/f2e92653-4db8-442d-b937-f84fa7b5e787-kube-api-access-jqwt4\") pod \"f2e92653-4db8-442d-b937-f84fa7b5e787\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") "
Sep 16 14:25:05.659379 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659266    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2e92653-4db8-442d-b937-f84fa7b5e787-tls-certs\") pod \"f2e92653-4db8-442d-b937-f84fa7b5e787\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") "
Sep 16 14:25:05.659379 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659293    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-dshm\") pod \"661395fe-d382-4970-bd6f-1cadca42c6fa\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") "
Sep 16 14:25:05.659490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659439    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-home\") pod \"f2e92653-4db8-442d-b937-f84fa7b5e787\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") "
Sep 16 14:25:05.659490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659472    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-tmp-dir\") pod \"929e9227-acf9-4f5f-9818-a6e6db35ed14\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") "
Sep 16 14:25:05.659607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659493    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-home\") pod \"661395fe-d382-4970-bd6f-1cadca42c6fa\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") "
Sep 16 14:25:05.659607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659524    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-dshm\") pod \"929e9227-acf9-4f5f-9818-a6e6db35ed14\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") "
Sep 16 14:25:05.659607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-tmp-dir\") pod \"661395fe-d382-4970-bd6f-1cadca42c6fa\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") "
Sep 16 14:25:05.659607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-model-cache\") pod \"929e9227-acf9-4f5f-9818-a6e6db35ed14\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") "
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659642    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-home\") pod \"929e9227-acf9-4f5f-9818-a6e6db35ed14\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") "
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-model-cache\") pod \"f2e92653-4db8-442d-b937-f84fa7b5e787\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") "
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659672    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-home" (OuterVolumeSpecName: "home") pod "f2e92653-4db8-442d-b937-f84fa7b5e787" (UID: "f2e92653-4db8-442d-b937-f84fa7b5e787"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659697    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-dshm\") pod \"f2e92653-4db8-442d-b937-f84fa7b5e787\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") "
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t7xt\" (UniqueName: \"kubernetes.io/projected/929e9227-acf9-4f5f-9818-a6e6db35ed14-kube-api-access-9t7xt\") pod \"929e9227-acf9-4f5f-9818-a6e6db35ed14\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") "
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659781    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-tmp-dir\") pod \"f2e92653-4db8-442d-b937-f84fa7b5e787\" (UID: \"f2e92653-4db8-442d-b937-f84fa7b5e787\") "
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-model-cache\") pod \"661395fe-d382-4970-bd6f-1cadca42c6fa\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") "
Sep 16 14:25:05.659868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659858    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2vp6\" (UniqueName: \"kubernetes.io/projected/661395fe-d382-4970-bd6f-1cadca42c6fa-kube-api-access-x2vp6\") pod \"661395fe-d382-4970-bd6f-1cadca42c6fa\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") "
Sep 16 14:25:05.660540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "661395fe-d382-4970-bd6f-1cadca42c6fa" (UID: "661395fe-d382-4970-bd6f-1cadca42c6fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.660540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659928    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/929e9227-acf9-4f5f-9818-a6e6db35ed14-tls-certs\") pod \"929e9227-acf9-4f5f-9818-a6e6db35ed14\" (UID: \"929e9227-acf9-4f5f-9818-a6e6db35ed14\") "
Sep 16 14:25:05.660540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.659956    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/661395fe-d382-4970-bd6f-1cadca42c6fa-tls-certs\") pod \"661395fe-d382-4970-bd6f-1cadca42c6fa\" (UID: \"661395fe-d382-4970-bd6f-1cadca42c6fa\") "
Sep 16 14:25:05.660540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.660183    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-home" (OuterVolumeSpecName: "home") pod "661395fe-d382-4970-bd6f-1cadca42c6fa" (UID: "661395fe-d382-4970-bd6f-1cadca42c6fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.660540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.660432    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.660540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.660453    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.660540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.660465    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.660906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.660663    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2e92653-4db8-442d-b937-f84fa7b5e787" (UID: "f2e92653-4db8-442d-b937-f84fa7b5e787"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.660906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.660670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "929e9227-acf9-4f5f-9818-a6e6db35ed14" (UID: "929e9227-acf9-4f5f-9818-a6e6db35ed14"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.661516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.661255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-home" (OuterVolumeSpecName: "home") pod "929e9227-acf9-4f5f-9818-a6e6db35ed14" (UID: "929e9227-acf9-4f5f-9818-a6e6db35ed14"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.661516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.661288    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2e92653-4db8-442d-b937-f84fa7b5e787" (UID: "f2e92653-4db8-442d-b937-f84fa7b5e787"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.661516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.661463    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-model-cache" (OuterVolumeSpecName: "model-cache") pod "929e9227-acf9-4f5f-9818-a6e6db35ed14" (UID: "929e9227-acf9-4f5f-9818-a6e6db35ed14"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.661516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.661496    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "661395fe-d382-4970-bd6f-1cadca42c6fa" (UID: "661395fe-d382-4970-bd6f-1cadca42c6fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.663040 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.663009    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2e92653-4db8-442d-b937-f84fa7b5e787-kube-api-access-jqwt4" (OuterVolumeSpecName: "kube-api-access-jqwt4") pod "f2e92653-4db8-442d-b937-f84fa7b5e787" (UID: "f2e92653-4db8-442d-b937-f84fa7b5e787"). InnerVolumeSpecName "kube-api-access-jqwt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:05.663160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.663052    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/661395fe-d382-4970-bd6f-1cadca42c6fa-kube-api-access-x2vp6" (OuterVolumeSpecName: "kube-api-access-x2vp6") pod "661395fe-d382-4970-bd6f-1cadca42c6fa" (UID: "661395fe-d382-4970-bd6f-1cadca42c6fa"). InnerVolumeSpecName "kube-api-access-x2vp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:05.663437 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.663415    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-dshm" (OuterVolumeSpecName: "dshm") pod "661395fe-d382-4970-bd6f-1cadca42c6fa" (UID: "661395fe-d382-4970-bd6f-1cadca42c6fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.663521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.663501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-dshm" (OuterVolumeSpecName: "dshm") pod "f2e92653-4db8-442d-b937-f84fa7b5e787" (UID: "f2e92653-4db8-442d-b937-f84fa7b5e787"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.663983 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.663950    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/661395fe-d382-4970-bd6f-1cadca42c6fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "661395fe-d382-4970-bd6f-1cadca42c6fa" (UID: "661395fe-d382-4970-bd6f-1cadca42c6fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:25:05.664339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.664314    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2e92653-4db8-442d-b937-f84fa7b5e787-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2e92653-4db8-442d-b937-f84fa7b5e787" (UID: "f2e92653-4db8-442d-b937-f84fa7b5e787"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:25:05.664710 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.664690    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/929e9227-acf9-4f5f-9818-a6e6db35ed14-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "929e9227-acf9-4f5f-9818-a6e6db35ed14" (UID: "929e9227-acf9-4f5f-9818-a6e6db35ed14"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:25:05.664812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.664799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-dshm" (OuterVolumeSpecName: "dshm") pod "929e9227-acf9-4f5f-9818-a6e6db35ed14" (UID: "929e9227-acf9-4f5f-9818-a6e6db35ed14"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.664981 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.664958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/929e9227-acf9-4f5f-9818-a6e6db35ed14-kube-api-access-9t7xt" (OuterVolumeSpecName: "kube-api-access-9t7xt") pod "929e9227-acf9-4f5f-9818-a6e6db35ed14" (UID: "929e9227-acf9-4f5f-9818-a6e6db35ed14"). InnerVolumeSpecName "kube-api-access-9t7xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:05.761857 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761828    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.761857 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761854    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.761857 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761863    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761872    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t7xt\" (UniqueName: \"kubernetes.io/projected/929e9227-acf9-4f5f-9818-a6e6db35ed14-kube-api-access-9t7xt\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761881    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2e92653-4db8-442d-b937-f84fa7b5e787-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761892    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761901    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2vp6\" (UniqueName: \"kubernetes.io/projected/661395fe-d382-4970-bd6f-1cadca42c6fa-kube-api-access-x2vp6\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761909    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/929e9227-acf9-4f5f-9818-a6e6db35ed14-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761916    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/661395fe-d382-4970-bd6f-1cadca42c6fa-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761924    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqwt4\" (UniqueName: \"kubernetes.io/projected/f2e92653-4db8-442d-b937-f84fa7b5e787-kube-api-access-jqwt4\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761931    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2e92653-4db8-442d-b937-f84fa7b5e787-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761938    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/661395fe-d382-4970-bd6f-1cadca42c6fa-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761944    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761951    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.762074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:05.761958    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/929e9227-acf9-4f5f-9818-a6e6db35ed14-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:25:06.079088 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.079060    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:06.200692 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.200622    2580 generic.go:356] "Generic (PLEG): container finished" podID="f2e92653-4db8-442d-b937-f84fa7b5e787" containerID="6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511" exitCode=0
Sep 16 14:25:06.200692 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.200665    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"
Sep 16 14:25:06.200827 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.200700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz" event={"ID":"f2e92653-4db8-442d-b937-f84fa7b5e787","Type":"ContainerDied","Data":"6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511"}
Sep 16 14:25:06.200827 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.200745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz" event={"ID":"f2e92653-4db8-442d-b937-f84fa7b5e787","Type":"ContainerDied","Data":"41c681504c309e722fe770cca73ddf41c99e3efade95d263ba62e517315b1d5a"}
Sep 16 14:25:06.200827 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.200763    2580 scope.go:117] "RemoveContainer" containerID="6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511"
Sep 16 14:25:06.202376 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.202337    2580 generic.go:356] "Generic (PLEG): container finished" podID="661395fe-d382-4970-bd6f-1cadca42c6fa" containerID="3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826" exitCode=0
Sep 16 14:25:06.202505 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.202393    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"
Sep 16 14:25:06.202505 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.202415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt" event={"ID":"661395fe-d382-4970-bd6f-1cadca42c6fa","Type":"ContainerDied","Data":"3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826"}
Sep 16 14:25:06.202668 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.202528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt" event={"ID":"661395fe-d382-4970-bd6f-1cadca42c6fa","Type":"ContainerDied","Data":"77276a15934537dce878711d63d30f305bc95b7f4506502db256a14f49cf897a"}
Sep 16 14:25:06.204357 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.204322    2580 generic.go:356] "Generic (PLEG): container finished" podID="929e9227-acf9-4f5f-9818-a6e6db35ed14" containerID="f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e" exitCode=0
Sep 16 14:25:06.204521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.204378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz" event={"ID":"929e9227-acf9-4f5f-9818-a6e6db35ed14","Type":"ContainerDied","Data":"f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e"}
Sep 16 14:25:06.204521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.204404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz" event={"ID":"929e9227-acf9-4f5f-9818-a6e6db35ed14","Type":"ContainerDied","Data":"415213b21422f52162c4404972e2fa6b170559371abcbe0a11a53dab297c0c90"}
Sep 16 14:25:06.204521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.204455    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"
Sep 16 14:25:06.210325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.210305    2580 scope.go:117] "RemoveContainer" containerID="6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511"
Sep 16 14:25:06.210641 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:25:06.210619    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511\": container with ID starting with 6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511 not found: ID does not exist" containerID="6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511"
Sep 16 14:25:06.210736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.210652    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511"} err="failed to get container status \"6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511\": rpc error: code = NotFound desc = could not find container \"6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511\": container with ID starting with 6817e8dbe1760a0d2045ec70b3b9b14edbd51c54a6e17b4e912134f1c1654511 not found: ID does not exist"
Sep 16 14:25:06.210736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.210668    2580 scope.go:117] "RemoveContainer" containerID="3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826"
Sep 16 14:25:06.219723 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.219706    2580 scope.go:117] "RemoveContainer" containerID="3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826"
Sep 16 14:25:06.220003 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:25:06.219986    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826\": container with ID starting with 3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826 not found: ID does not exist" containerID="3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826"
Sep 16 14:25:06.220062 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.220010    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826"} err="failed to get container status \"3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826\": rpc error: code = NotFound desc = could not find container \"3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826\": container with ID starting with 3d1e0c7b3fa75f34fb2356ae533a3c22ee800f45d9e6aaf8304e0398882d3826 not found: ID does not exist"
Sep 16 14:25:06.220062 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.220030    2580 scope.go:117] "RemoveContainer" containerID="f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e"
Sep 16 14:25:06.225982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.225960    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"]
Sep 16 14:25:06.227605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.227591    2580 scope.go:117] "RemoveContainer" containerID="f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e"
Sep 16 14:25:06.227859 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:25:06.227845    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e\": container with ID starting with f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e not found: ID does not exist" containerID="f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e"
Sep 16 14:25:06.227904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.227864    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e"} err="failed to get container status \"f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e\": rpc error: code = NotFound desc = could not find container \"f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e\": container with ID starting with f5322224eeafa79f86c62922a18293295196efff122dea002d8b6ca821c9860e not found: ID does not exist"
Sep 16 14:25:06.237489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.237469    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/premium-w3-2ad565-kserve-774ccb749b-2tfwt"]
Sep 16 14:25:06.252291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.252272    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"]
Sep 16 14:25:06.257905 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.257876    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-2-w3-2ad565-kserve-759b7b94dd-v8nqz"]
Sep 16 14:25:06.272171 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.272153    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"]
Sep 16 14:25:06.280797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:06.280779    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/simulator-w3-2ad565-kserve-58ff6795fc-5zjfz"]
Sep 16 14:25:08.007671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:08.007631    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="661395fe-d382-4970-bd6f-1cadca42c6fa" path="/var/lib/kubelet/pods/661395fe-d382-4970-bd6f-1cadca42c6fa/volumes"
Sep 16 14:25:08.008074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:08.008021    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="929e9227-acf9-4f5f-9818-a6e6db35ed14" path="/var/lib/kubelet/pods/929e9227-acf9-4f5f-9818-a6e6db35ed14/volumes"
Sep 16 14:25:08.008421 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:08.008403    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2e92653-4db8-442d-b937-f84fa7b5e787" path="/var/lib/kubelet/pods/f2e92653-4db8-442d-b937-f84fa7b5e787/volumes"
Sep 16 14:25:09.208381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:09.208344    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:09.276799 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:09.276774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:09.679571 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:09.679480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:20.544456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:20.544426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:24.126414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:24.126336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:25:28.637949 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:28.637911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:28.786699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:28.786670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:29.088922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.088897    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"]
Sep 16 14:25:29.089355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089340    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="929e9227-acf9-4f5f-9818-a6e6db35ed14" containerName="main"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089357    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="929e9227-acf9-4f5f-9818-a6e6db35ed14" containerName="main"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089366    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="661395fe-d382-4970-bd6f-1cadca42c6fa" containerName="main"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089371    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="661395fe-d382-4970-bd6f-1cadca42c6fa" containerName="main"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089380    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2e92653-4db8-442d-b937-f84fa7b5e787" containerName="main"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089385    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2e92653-4db8-442d-b937-f84fa7b5e787" containerName="main"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089395    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f98486f-75e8-4bb6-9e7d-842a485268f4" containerName="payload-processing"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089401    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f98486f-75e8-4bb6-9e7d-842a485268f4" containerName="payload-processing"
Sep 16 14:25:29.089413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089410    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f8a28f-e5f9-48b6-9a04-122413c04286" containerName="revoke-keys"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089417    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f8a28f-e5f9-48b6-9a04-122413c04286" containerName="revoke-keys"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089435    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" containerName="payload-pre-processing"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089440    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" containerName="payload-pre-processing"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089501    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f98486f-75e8-4bb6-9e7d-842a485268f4" containerName="payload-processing"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089510    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2e92653-4db8-442d-b937-f84fa7b5e787" containerName="main"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089521    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="661395fe-d382-4970-bd6f-1cadca42c6fa" containerName="main"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089526    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e05fdf3-4dab-4b4e-ae6f-bd37d0b37b4d" containerName="payload-pre-processing"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089531    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="929e9227-acf9-4f5f-9818-a6e6db35ed14" containerName="main"
Sep 16 14:25:29.089654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.089536    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f8a28f-e5f9-48b6-9a04-122413c04286" containerName="revoke-keys"
Sep 16 14:25:29.092558 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.092544    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.096311 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.096291    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"embedding-w0-08475e-kserve-self-signed-certs\""
Sep 16 14:25:29.116403 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.116383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"]
Sep 16 14:25:29.158855 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.158828    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:29.165841 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.165817    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7x9g\" (UniqueName: \"kubernetes.io/projected/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-kube-api-access-r7x9g\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.165917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.165863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-model-cache\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.165961 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.165919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tls-certs\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.165961 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.165941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tmp-dir\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.166025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.165968    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-dshm\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.166025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.166013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-home\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.266767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.266736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tmp-dir\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.266767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.266768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-dshm\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.266902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.266810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-home\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.266902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.266831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7x9g\" (UniqueName: \"kubernetes.io/projected/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-kube-api-access-r7x9g\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.266902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.266866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-model-cache\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.266902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.266896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tls-certs\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.267112 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.267092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tmp-dir\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.267200 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.267182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-home\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.267244 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.267208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-model-cache\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.269094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.269067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-dshm\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.269307 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.269289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tls-certs\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.295375 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.295355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7x9g\" (UniqueName: \"kubernetes.io/projected/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-kube-api-access-r7x9g\") pod \"embedding-w0-08475e-kserve-778d9dd677-ff2jb\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") " pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.400970 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.400915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:29.694431 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.694407    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"]
Sep 16 14:25:29.696562 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:25:29.696535    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd6c1f80_4f92_4bdd_a30a_6a24cb47a550.slice/crio-ac05dc4162d0c763be210241cc998ae296c28f8051c85a15133b5303fb57cc39 WatchSource:0}: Error finding container ac05dc4162d0c763be210241cc998ae296c28f8051c85a15133b5303fb57cc39: Status 404 returned error can't find the container with id ac05dc4162d0c763be210241cc998ae296c28f8051c85a15133b5303fb57cc39
Sep 16 14:25:29.765854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:29.765823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:30.153867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:30.153841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:30.279778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:30.279752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:30.336078 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:30.336023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb" event={"ID":"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550","Type":"ContainerStarted","Data":"efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d"}
Sep 16 14:25:30.336078 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:30.336055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb" event={"ID":"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550","Type":"ContainerStarted","Data":"ac05dc4162d0c763be210241cc998ae296c28f8051c85a15133b5303fb57cc39"}
Sep 16 14:25:30.369284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:30.369241    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb" podStartSLOduration=2.369226796 podStartE2EDuration="2.369226796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:25:30.367972004 +0000 UTC m=+1260.855982080" watchObservedRunningTime="2026-09-16 14:25:30.369226796 +0000 UTC m=+1260.857236910"
Sep 16 14:25:30.393656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:30.393631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:31.331795 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:31.331759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:39.401641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:39.401603    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:39.402048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:39.401659    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:39.414020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:39.413994    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:40.409600 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:40.409562    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:25:45.025369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:45.025328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:45.818185 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:45.818129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:46.474074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:46.474038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:25:50.891284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:50.891243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:51.549000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:51.548955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:52.181103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:52.181070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:25:54.000377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:25:54.000338    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:26:01.560906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:01.560869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:02.317243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:02.317214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:07.095003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.094965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:26:07.171514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.171478    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"]
Sep 16 14:26:07.175510 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.175486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.178383 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.178358    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"default-dockercfg-bz4h4\""
Sep 16 14:26:07.178503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.178391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"auth-test-model-f72402-kserve-self-signed-certs\""
Sep 16 14:26:07.179347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.179328    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"kube-root-ca.crt\""
Sep 16 14:26:07.179462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.179365    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"openshift-service-ca.crt\""
Sep 16 14:26:07.189286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.189266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"]
Sep 16 14:26:07.216861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.216834    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:07.277006 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.276983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tls-certs\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.277095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.277021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-home\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.277095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.277061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srrmr\" (UniqueName: \"kubernetes.io/projected/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-kube-api-access-srrmr\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.277095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.277088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tmp-dir\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.277227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.277111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-model-cache\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.277227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.277205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-dshm\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.377753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.377699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srrmr\" (UniqueName: \"kubernetes.io/projected/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-kube-api-access-srrmr\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.377753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.377726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tmp-dir\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.377900 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.377754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-model-cache\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.377900 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.377781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-dshm\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.377900 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.377812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tls-certs\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.377900 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.377863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-home\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.378080 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.378058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tmp-dir\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.378162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.378114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-model-cache\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.378234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.378217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-home\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.380026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.380004    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-dshm\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.380331 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.380312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tls-certs\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.391488 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.391465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srrmr\" (UniqueName: \"kubernetes.io/projected/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-kube-api-access-srrmr\") pod \"auth-test-model-f72402-kserve-58df7b887f-625zg\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") " pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.487881 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.487857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:07.761411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:07.761385    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"]
Sep 16 14:26:07.762952 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:26:07.762928    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89e8c2f0_212e_4ba5_88d7_5ffcb53ad23e.slice/crio-fab9fa4f65304cbd61cb4374d1bef908cdb12b7c7a77ad31761cd64f20d35b00 WatchSource:0}: Error finding container fab9fa4f65304cbd61cb4374d1bef908cdb12b7c7a77ad31761cd64f20d35b00: Status 404 returned error can't find the container with id fab9fa4f65304cbd61cb4374d1bef908cdb12b7c7a77ad31761cd64f20d35b00
Sep 16 14:26:08.558540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:08.558498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg" event={"ID":"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e","Type":"ContainerStarted","Data":"612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a"}
Sep 16 14:26:08.558936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:08.558557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg" event={"ID":"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e","Type":"ContainerStarted","Data":"fab9fa4f65304cbd61cb4374d1bef908cdb12b7c7a77ad31761cd64f20d35b00"}
Sep 16 14:26:08.588016 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:08.587972    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg" podStartSLOduration=1.587955701 podStartE2EDuration="1.587955701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:08.587944057 +0000 UTC m=+1299.075954132" watchObservedRunningTime="2026-09-16 14:26:08.587955701 +0000 UTC m=+1299.075965774"
Sep 16 14:26:09.439172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:09.439126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:09.688951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:09.688916    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:09.917933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:09.917898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:10.515989 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:10.515961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:10.610917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:10.610872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:12.261376 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:12.261330    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:13.712556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:13.712483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:13.911550 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:13.911522    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"]
Sep 16 14:26:13.915179 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:13.915160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"
Sep 16 14:26:13.919902 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:13.919879    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:26:13.920019 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:13.919942    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:26:13.955753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:13.955735    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"]
Sep 16 14:26:14.041979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:14.041922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbd68e7-17c9-4f46-a002-ab10c9c31f3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6\" (UID: \"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"
Sep 16 14:26:14.143156 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:14.143104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbd68e7-17c9-4f46-a002-ab10c9c31f3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6\" (UID: \"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"
Sep 16 14:26:14.143761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:14.143742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbd68e7-17c9-4f46-a002-ab10c9c31f3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6\" (UID: \"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"
Sep 16 14:26:14.226674 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:14.226649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"
Sep 16 14:26:14.299095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:14.299027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:14.511974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:14.511932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"]
Sep 16 14:26:14.514397 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:26:14.514366    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dbd68e7_17c9_4f46_a002_ab10c9c31f3b.slice/crio-aa3c3612718a41e5aa4e31f5f7e98fc6036005a7778a67c84c524835ff4c66bf WatchSource:0}: Error finding container aa3c3612718a41e5aa4e31f5f7e98fc6036005a7778a67c84c524835ff4c66bf: Status 404 returned error can't find the container with id aa3c3612718a41e5aa4e31f5f7e98fc6036005a7778a67c84c524835ff4c66bf
Sep 16 14:26:14.600008 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:14.599980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6" event={"ID":"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b","Type":"ContainerStarted","Data":"aa3c3612718a41e5aa4e31f5f7e98fc6036005a7778a67c84c524835ff4c66bf"}
Sep 16 14:26:15.353082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:15.353050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:15.607035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:15.606963    2580 generic.go:356] "Generic (PLEG): container finished" podID="4dbd68e7-17c9-4f46-a002-ab10c9c31f3b" containerID="b8b48ec89d86973e4d2b384e8033e32c328d04603d42d693b0b37aafa293ad2f" exitCode=0
Sep 16 14:26:15.607172 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:15.607045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6" event={"ID":"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b","Type":"ContainerDied","Data":"b8b48ec89d86973e4d2b384e8033e32c328d04603d42d693b0b37aafa293ad2f"}
Sep 16 14:26:16.299084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:16.299048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:16.750629 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:16.750601    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"
Sep 16 14:26:16.864740 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:16.864674    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbd68e7-17c9-4f46-a002-ab10c9c31f3b-maas-api-service-ca\") pod \"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b\" (UID: \"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b\") "
Sep 16 14:26:16.864970 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:16.864949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dbd68e7-17c9-4f46-a002-ab10c9c31f3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4dbd68e7-17c9-4f46-a002-ab10c9c31f3b" (UID: "4dbd68e7-17c9-4f46-a002-ab10c9c31f3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:16.865077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:16.865060    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbd68e7-17c9-4f46-a002-ab10c9c31f3b-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:17.121331 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.121267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:17.488335 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.488304    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:17.488487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.488350    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:17.500657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.500637    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:17.620060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.620037    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"
Sep 16 14:26:17.620235 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.620077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6" event={"ID":"4dbd68e7-17c9-4f46-a002-ab10c9c31f3b","Type":"ContainerDied","Data":"aa3c3612718a41e5aa4e31f5f7e98fc6036005a7778a67c84c524835ff4c66bf"}
Sep 16 14:26:17.620235 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.620122    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa3c3612718a41e5aa4e31f5f7e98fc6036005a7778a67c84c524835ff4c66bf"
Sep 16 14:26:17.631515 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:17.631494    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:26:18.692327 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:18.692291    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"]
Sep 16 14:26:18.703246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:18.703220    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"]
Sep 16 14:26:18.979744 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:18.979709    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:19.000153 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:19.000115    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:26:19.239479 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:19.239411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:19.359030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:19.359004    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:20.000545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:20.000512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:25.938594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:25.938554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:26.038288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.038258    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:26.173734 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.173699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:26.553437 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.553401    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"]
Sep 16 14:26:26.553895 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.553878    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dbd68e7-17c9-4f46-a002-ab10c9c31f3b" containerName="revoke-keys"
Sep 16 14:26:26.554006 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.553897    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dbd68e7-17c9-4f46-a002-ab10c9c31f3b" containerName="revoke-keys"
Sep 16 14:26:26.554066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.554004    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dbd68e7-17c9-4f46-a002-ab10c9c31f3b" containerName="revoke-keys"
Sep 16 14:26:26.556543 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.556523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.560991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.560966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dbb84810-openshift-default-dockercfg-d78c8\""
Sep 16 14:26:26.561303 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.561277    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 14:26:26.611657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.611628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"]
Sep 16 14:26:26.644814 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.644788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-credential-socket\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.644922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.644829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-envoy\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.644922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.644853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71458779-7755-46c9-887b-0e63ababad22-istiod-ca-cert\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.644922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.644877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-istio-token\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.644922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.644897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-certs\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.644922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.644914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvzfs\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-kube-api-access-vvzfs\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.645121 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.645060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71458779-7755-46c9-887b-0e63ababad22-istio-podinfo\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.645201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.645124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-socket\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.645201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.645182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-data\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.664208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.664179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:26.746130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71458779-7755-46c9-887b-0e63ababad22-istio-podinfo\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-socket\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746194    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-data\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-credential-socket\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-envoy\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71458779-7755-46c9-887b-0e63ababad22-istiod-ca-cert\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-istio-token\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-certs\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746608 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvzfs\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-kube-api-access-vvzfs\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746787 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-socket\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746787 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-data\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746787 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-credential-socket\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.746998 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.746940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-certs\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.747085 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.747066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71458779-7755-46c9-887b-0e63ababad22-istiod-ca-cert\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.748467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.748449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-envoy\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.748591 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.748575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71458779-7755-46c9-887b-0e63ababad22-istio-podinfo\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.766120 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.766098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvzfs\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-kube-api-access-vvzfs\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.766304 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.766288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-istio-token\") pod \"e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") " pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:26.836840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.836781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:26.866497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:26.866471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:27.148926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:27.148831    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"]
Sep 16 14:26:27.151647 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:26:27.151622    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71458779_7755_46c9_887b_0e63ababad22.slice/crio-4a6b9bfad52dd6f30281453fefef28207d6acdf140603a06f12da781bd1c5cad WatchSource:0}: Error finding container 4a6b9bfad52dd6f30281453fefef28207d6acdf140603a06f12da781bd1c5cad: Status 404 returned error can't find the container with id 4a6b9bfad52dd6f30281453fefef28207d6acdf140603a06f12da781bd1c5cad
Sep 16 14:26:27.153938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:27.153908    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:26:27.154028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:27.153969    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:26:27.154028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:27.153995    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:26:27.675251 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:27.675219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" event={"ID":"71458779-7755-46c9-887b-0e63ababad22","Type":"ContainerStarted","Data":"84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4"}
Sep 16 14:26:27.675421 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:27.675267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" event={"ID":"71458779-7755-46c9-887b-0e63ababad22","Type":"ContainerStarted","Data":"4a6b9bfad52dd6f30281453fefef28207d6acdf140603a06f12da781bd1c5cad"}
Sep 16 14:26:27.866689 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:27.866664    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:28.873458 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:28.873423    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:28.905936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:28.905874    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podStartSLOduration=2.905857381 podStartE2EDuration="2.905857381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:27.708408455 +0000 UTC m=+1318.196418552" watchObservedRunningTime="2026-09-16 14:26:28.905857381 +0000 UTC m=+1319.393867453"
Sep 16 14:26:29.630048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:29.630017    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:29.686288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:29.686261    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:29.687783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:29.687760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:29.870798 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:29.870765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:29.977817 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:29.977782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:30.493939 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:30.493905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:31.787847 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:31.787818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:31.861201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:31.861162    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"]
Sep 16 14:26:31.861410 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:31.861373    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc" podUID="4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" containerName="payload-processing" containerID="cri-o://8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172" gracePeriod=30
Sep 16 14:26:31.898931 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:31.898893    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"]
Sep 16 14:26:31.899180 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:31.899151    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929" podUID="436fef94-26bf-4af1-8b9c-22dfa97c6d53" containerName="payload-pre-processing" containerID="cri-o://05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c" gracePeriod=30
Sep 16 14:26:32.065418 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.065366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:32.147945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.147919    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:26:32.167884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.167861    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:26:32.296045 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.296017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gggm\" (UniqueName: \"kubernetes.io/projected/436fef94-26bf-4af1-8b9c-22dfa97c6d53-kube-api-access-9gggm\") pod \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") "
Sep 16 14:26:32.296214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.296164    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/436fef94-26bf-4af1-8b9c-22dfa97c6d53-plugins-config-volume\") pod \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\" (UID: \"436fef94-26bf-4af1-8b9c-22dfa97c6d53\") "
Sep 16 14:26:32.296214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.296205    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-plugins-config-volume\") pod \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") "
Sep 16 14:26:32.296312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.296231    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxxlc\" (UniqueName: \"kubernetes.io/projected/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-kube-api-access-gxxlc\") pod \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\" (UID: \"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da\") "
Sep 16 14:26:32.296580 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.296549    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/436fef94-26bf-4af1-8b9c-22dfa97c6d53-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "436fef94-26bf-4af1-8b9c-22dfa97c6d53" (UID: "436fef94-26bf-4af1-8b9c-22dfa97c6d53"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:32.296580 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.296551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" (UID: "4434d2a7-1c12-4eaa-902c-ef5f3a0a35da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:32.298149 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.298111    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/436fef94-26bf-4af1-8b9c-22dfa97c6d53-kube-api-access-9gggm" (OuterVolumeSpecName: "kube-api-access-9gggm") pod "436fef94-26bf-4af1-8b9c-22dfa97c6d53" (UID: "436fef94-26bf-4af1-8b9c-22dfa97c6d53"). InnerVolumeSpecName "kube-api-access-9gggm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:32.298225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.298133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-kube-api-access-gxxlc" (OuterVolumeSpecName: "kube-api-access-gxxlc") pod "4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" (UID: "4434d2a7-1c12-4eaa-902c-ef5f3a0a35da"). InnerVolumeSpecName "kube-api-access-gxxlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:32.301624 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.301604    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:32.397908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.397854    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/436fef94-26bf-4af1-8b9c-22dfa97c6d53-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.397908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.397875    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.397908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.397884    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxxlc\" (UniqueName: \"kubernetes.io/projected/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da-kube-api-access-gxxlc\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.397908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.397892    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gggm\" (UniqueName: \"kubernetes.io/projected/436fef94-26bf-4af1-8b9c-22dfa97c6d53-kube-api-access-9gggm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.511382 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.511359    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"]
Sep 16 14:26:32.523865 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.523840    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:32.525641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.525623    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-08475e-dfnm6"]
Sep 16 14:26:32.703465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.703401    2580 generic.go:356] "Generic (PLEG): container finished" podID="4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" containerID="8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172" exitCode=0
Sep 16 14:26:32.703465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.703441    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"
Sep 16 14:26:32.703611 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.703481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc" event={"ID":"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da","Type":"ContainerDied","Data":"8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172"}
Sep 16 14:26:32.703611 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.703535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc" event={"ID":"4434d2a7-1c12-4eaa-902c-ef5f3a0a35da","Type":"ContainerDied","Data":"cb8f3b9a2ae99f5be7c272241a52a2d7660a9b1976b4d6d6eb2350bb0e57747f"}
Sep 16 14:26:32.703611 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.703570    2580 scope.go:117] "RemoveContainer" containerID="8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172"
Sep 16 14:26:32.704911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.704875    2580 generic.go:356] "Generic (PLEG): container finished" podID="436fef94-26bf-4af1-8b9c-22dfa97c6d53" containerID="05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c" exitCode=0
Sep 16 14:26:32.705051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.704916    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"
Sep 16 14:26:32.705051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.704954    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929" event={"ID":"436fef94-26bf-4af1-8b9c-22dfa97c6d53","Type":"ContainerDied","Data":"05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c"}
Sep 16 14:26:32.705051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.704999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929" event={"ID":"436fef94-26bf-4af1-8b9c-22dfa97c6d53","Type":"ContainerDied","Data":"b1f288c3167f51fa8d4e72b218e5bd4259ee26c683042faff1a83495867629c1"}
Sep 16 14:26:32.713713 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.713692    2580 scope.go:117] "RemoveContainer" containerID="8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172"
Sep 16 14:26:32.713999 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:26:32.713979    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172\": container with ID starting with 8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172 not found: ID does not exist" containerID="8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172"
Sep 16 14:26:32.714070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.714005    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172"} err="failed to get container status \"8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172\": rpc error: code = NotFound desc = could not find container \"8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172\": container with ID starting with 8c43ac55dfdd24773e7bb26644ebf91f43d7d26436f216573a3c724d24cc6172 not found: ID does not exist"
Sep 16 14:26:32.714070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.714022    2580 scope.go:117] "RemoveContainer" containerID="05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c"
Sep 16 14:26:32.721724 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.721707    2580 scope.go:117] "RemoveContainer" containerID="05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c"
Sep 16 14:26:32.721959 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:26:32.721942    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c\": container with ID starting with 05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c not found: ID does not exist" containerID="05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c"
Sep 16 14:26:32.722017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.721965    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c"} err="failed to get container status \"05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c\": rpc error: code = NotFound desc = could not find container \"05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c\": container with ID starting with 05fffee742e5fbc9c9157feab53c514f349361fb69b1011c11462ea6c65b3b9c not found: ID does not exist"
Sep 16 14:26:32.752533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.752509    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"]
Sep 16 14:26:32.772259 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.772238    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-08475e-6bfd87f97-kklgc"]
Sep 16 14:26:32.825525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.825507    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"]
Sep 16 14:26:32.831716 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.831692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:32.846850 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:32.846829    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-08475e-5cdfd989db-kr929"]
Sep 16 14:26:33.024868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.024846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:33.711873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.711836    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t" podUID="c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" containerName="main" containerID="cri-o://6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6" gracePeriod=45
Sep 16 14:26:33.723478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.723453    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb" podUID="dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" containerName="main" containerID="cri-o://efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d" gracePeriod=45
Sep 16 14:26:33.787791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.787764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:26:33.873708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.873672    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:33.979040 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.979011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:33.992200 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.992181    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:26:33.995359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:33.995340    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:26:34.005854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.005833    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="436fef94-26bf-4af1-8b9c-22dfa97c6d53" path="/var/lib/kubelet/pods/436fef94-26bf-4af1-8b9c-22dfa97c6d53/volumes"
Sep 16 14:26:34.006180 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.006168    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" path="/var/lib/kubelet/pods/4434d2a7-1c12-4eaa-902c-ef5f3a0a35da/volumes"
Sep 16 14:26:34.006451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.006440    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dbd68e7-17c9-4f46-a002-ab10c9c31f3b" path="/var/lib/kubelet/pods/4dbd68e7-17c9-4f46-a002-ab10c9c31f3b/volumes"
Sep 16 14:26:34.079078 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079045    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"]
Sep 16 14:26:34.079505 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079489    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" containerName="main"
Sep 16 14:26:34.079590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079507    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" containerName="main"
Sep 16 14:26:34.079590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079531    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fef94-26bf-4af1-8b9c-22dfa97c6d53" containerName="payload-pre-processing"
Sep 16 14:26:34.079590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079540    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fef94-26bf-4af1-8b9c-22dfa97c6d53" containerName="payload-pre-processing"
Sep 16 14:26:34.079590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079550    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" containerName="payload-processing"
Sep 16 14:26:34.079590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079559    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" containerName="payload-processing"
Sep 16 14:26:34.079590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079587    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" containerName="main"
Sep 16 14:26:34.079880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079596    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" containerName="main"
Sep 16 14:26:34.079880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079682    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" containerName="main"
Sep 16 14:26:34.079880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079695    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4434d2a7-1c12-4eaa-902c-ef5f3a0a35da" containerName="payload-processing"
Sep 16 14:26:34.079880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079707    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" containerName="main"
Sep 16 14:26:34.079880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.079715    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="436fef94-26bf-4af1-8b9c-22dfa97c6d53" containerName="payload-pre-processing"
Sep 16 14:26:34.083231 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.083212    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.086898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.086880    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-dbb84810\""
Sep 16 14:26:34.099176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.099152    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-dbb84810-dockercfg-jcnpc\""
Sep 16 14:26:34.106772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.106737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"]
Sep 16 14:26:34.113223 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113201    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-model-cache\") pod \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") "
Sep 16 14:26:34.113336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113259    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tls-certs\") pod \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") "
Sep 16 14:26:34.113336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113297    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-home\") pod \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") "
Sep 16 14:26:34.113336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113332    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tmp-dir\") pod \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") "
Sep 16 14:26:34.113503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-dshm\") pod \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") "
Sep 16 14:26:34.113503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-model-cache\") pod \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") "
Sep 16 14:26:34.113503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wkzw\" (UniqueName: \"kubernetes.io/projected/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-kube-api-access-8wkzw\") pod \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") "
Sep 16 14:26:34.113503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tmp-dir\") pod \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") "
Sep 16 14:26:34.113503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113470    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" (UID: "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.113785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-home\") pod \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") "
Sep 16 14:26:34.113785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7x9g\" (UniqueName: \"kubernetes.io/projected/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-kube-api-access-r7x9g\") pod \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") "
Sep 16 14:26:34.113785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" (UID: "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.113785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113624    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tls-certs\") pod \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\" (UID: \"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550\") "
Sep 16 14:26:34.113785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113656    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-dshm\") pod \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\" (UID: \"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4\") "
Sep 16 14:26:34.113785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" (UID: "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.113785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113771    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" (UID: "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.114161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.113951    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-home" (OuterVolumeSpecName: "home") pod "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" (UID: "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.114161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.114085    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.114161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.114105    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.114161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.114117    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.114161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.114130    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.114161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.114158    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.114455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.114395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-home" (OuterVolumeSpecName: "home") pod "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" (UID: "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.115988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.115959    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" (UID: "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:34.116604 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.116583    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-kube-api-access-r7x9g" (OuterVolumeSpecName: "kube-api-access-r7x9g") pod "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" (UID: "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550"). InnerVolumeSpecName "kube-api-access-r7x9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:34.116858 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.116834    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-dshm" (OuterVolumeSpecName: "dshm") pod "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" (UID: "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.116984 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.116967    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-dshm" (OuterVolumeSpecName: "dshm") pod "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" (UID: "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.117436 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.117403    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" (UID: "dd6c1f80-4f92-4bdd-a30a-6a24cb47a550"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:34.117529 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.117492    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-kube-api-access-8wkzw" (OuterVolumeSpecName: "kube-api-access-8wkzw") pod "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" (UID: "c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4"). InnerVolumeSpecName "kube-api-access-8wkzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:34.215194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r9lm\" (UniqueName: \"kubernetes.io/projected/6ff2b4c8-d298-4165-a639-253aafbc6abc-kube-api-access-4r9lm\") pod \"payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.215332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff2b4c8-d298-4165-a639-253aafbc6abc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.215332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215278    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.215332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215289    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.215332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215299    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.215332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215310    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.215332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215319    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.215332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215327    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wkzw\" (UniqueName: \"kubernetes.io/projected/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4-kube-api-access-8wkzw\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.215600 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.215335    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7x9g\" (UniqueName: \"kubernetes.io/projected/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550-kube-api-access-r7x9g\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.316285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.316248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff2b4c8-d298-4165-a639-253aafbc6abc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.316428 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.316355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r9lm\" (UniqueName: \"kubernetes.io/projected/6ff2b4c8-d298-4165-a639-253aafbc6abc-kube-api-access-4r9lm\") pod \"payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.316988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.316969    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff2b4c8-d298-4165-a639-253aafbc6abc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.330332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.330305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r9lm\" (UniqueName: \"kubernetes.io/projected/6ff2b4c8-d298-4165-a639-253aafbc6abc-kube-api-access-4r9lm\") pod \"payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.394601 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.394575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:34.677218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.677194    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"]
Sep 16 14:26:34.678977 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:26:34.678950    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff2b4c8_d298_4165_a639_253aafbc6abc.slice/crio-fc991d423bcca64d53b23f593c3b6264fc43065ddcaf02c4b63c001417c3c6ac WatchSource:0}: Error finding container fc991d423bcca64d53b23f593c3b6264fc43065ddcaf02c4b63c001417c3c6ac: Status 404 returned error can't find the container with id fc991d423bcca64d53b23f593c3b6264fc43065ddcaf02c4b63c001417c3c6ac
Sep 16 14:26:34.721300 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.721274    2580 generic.go:356] "Generic (PLEG): container finished" podID="dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" containerID="efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d" exitCode=0
Sep 16 14:26:34.721389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.721318    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"
Sep 16 14:26:34.721389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.721355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb" event={"ID":"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550","Type":"ContainerDied","Data":"efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d"}
Sep 16 14:26:34.721489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.721390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb" event={"ID":"dd6c1f80-4f92-4bdd-a30a-6a24cb47a550","Type":"ContainerDied","Data":"ac05dc4162d0c763be210241cc998ae296c28f8051c85a15133b5303fb57cc39"}
Sep 16 14:26:34.721489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.721407    2580 scope.go:117] "RemoveContainer" containerID="efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d"
Sep 16 14:26:34.722896 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.722871    2580 generic.go:356] "Generic (PLEG): container finished" podID="c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" containerID="6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6" exitCode=0
Sep 16 14:26:34.722985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.722965    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"
Sep 16 14:26:34.723043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.723007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t" event={"ID":"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4","Type":"ContainerDied","Data":"6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6"}
Sep 16 14:26:34.723174 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.723049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t" event={"ID":"c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4","Type":"ContainerDied","Data":"f92d88354bac338a390d716f740141de5b56c7a8d91392904af8994b1cca72fd"}
Sep 16 14:26:34.724429 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.724407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg" event={"ID":"6ff2b4c8-d298-4165-a639-253aafbc6abc","Type":"ContainerStarted","Data":"fc991d423bcca64d53b23f593c3b6264fc43065ddcaf02c4b63c001417c3c6ac"}
Sep 16 14:26:34.734913 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.734229    2580 scope.go:117] "RemoveContainer" containerID="efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d"
Sep 16 14:26:34.735031 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:26:34.734979    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d\": container with ID starting with efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d not found: ID does not exist" containerID="efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d"
Sep 16 14:26:34.735031 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.735012    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d"} err="failed to get container status \"efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d\": rpc error: code = NotFound desc = could not find container \"efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d\": container with ID starting with efb344e158820ea8543af8246335154ef2c0fcef0aaa3bdf5b54afde5324de9d not found: ID does not exist"
Sep 16 14:26:34.735604 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.735036    2580 scope.go:117] "RemoveContainer" containerID="6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6"
Sep 16 14:26:34.743818 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.743800    2580 scope.go:117] "RemoveContainer" containerID="6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6"
Sep 16 14:26:34.744067 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:26:34.744047    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6\": container with ID starting with 6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6 not found: ID does not exist" containerID="6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6"
Sep 16 14:26:34.744114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.744077    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6"} err="failed to get container status \"6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6\": rpc error: code = NotFound desc = could not find container \"6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6\": container with ID starting with 6f1027203a2645938c65560e38c4c6364afc510a82555b9148e9a3e7924aeea6 not found: ID does not exist"
Sep 16 14:26:34.806819 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.806797    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"]
Sep 16 14:26:34.808478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.808461    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/embedding-w0-08475e-kserve-778d9dd677-ff2jb"]
Sep 16 14:26:34.834736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.834706    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"]
Sep 16 14:26:34.839686 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.839664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:34.846683 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:34.846663    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-w0-08475e-kserve-94676f9c6-tzr6t"]
Sep 16 14:26:35.733583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:35.733555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg" event={"ID":"6ff2b4c8-d298-4165-a639-253aafbc6abc","Type":"ContainerStarted","Data":"4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4"}
Sep 16 14:26:35.733953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:35.733657    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:35.781431 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:35.781388    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg" podStartSLOduration=1.781375439 podStartE2EDuration="1.781375439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:35.778094164 +0000 UTC m=+1326.266104241" watchObservedRunningTime="2026-09-16 14:26:35.781375439 +0000 UTC m=+1326.269385514"
Sep 16 14:26:36.007937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:36.007868    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4" path="/var/lib/kubelet/pods/c332bc40-3330-4a2d-8f2a-e7c8a40d1ff4/volumes"
Sep 16 14:26:36.008275 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:36.008262    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd6c1f80-4f92-4bdd-a30a-6a24cb47a550" path="/var/lib/kubelet/pods/dd6c1f80-4f92-4bdd-a30a-6a24cb47a550/volumes"
Sep 16 14:26:36.413426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:36.413326    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:37.003389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:37.003354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:37.096940 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:37.096903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:37.504360 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:37.504332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:39.551176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:39.551121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:39.646392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:39.646357    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:39.821329 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:39.821244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:46.741472 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:46.741444    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:26:56.301507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:56.301447    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:56.684757 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:56.684671    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:56.830048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:56.830008    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:57.256984 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:57.256934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:57.915732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:57.915695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:58.061946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.061913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:58.172066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.171984    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:58.863577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.863539    2580 generic.go:356] "Generic (PLEG): container finished" podID="71458779-7755-46c9-887b-0e63ababad22" containerID="84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4" exitCode=0
Sep 16 14:26:58.863577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.863571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" event={"ID":"71458779-7755-46c9-887b-0e63ababad22","Type":"ContainerDied","Data":"84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4"}
Sep 16 14:26:58.863936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.863918    2580 scope.go:117] "RemoveContainer" containerID="84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4"
Sep 16 14:26:58.864933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.864902    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:26:58.865029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.864982    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:26:58.865092 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.865030    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:26:58.987020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:58.986991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:59.238574 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:59.238478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:26:59.686763 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:59.686687    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:26:59.870924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:26:59.870891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" event={"ID":"71458779-7755-46c9-887b-0e63ababad22","Type":"ContainerStarted","Data":"42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e"}
Sep 16 14:27:00.052413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:00.052380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:00.852255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:00.852221    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:00.867056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:00.867029    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:27:00.868181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:00.868159    2580 patch_prober.go:28] interesting pod/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:27:00.868285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:00.868206    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 16 14:27:01.170983 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:01.170894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:01.248947 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:01.248910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:27:01.320903 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:01.320872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:01.867785 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:01.867753    2580 patch_prober.go:28] interesting pod/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:27:01.867959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:01.867811    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 16 14:27:02.867313 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:02.867285    2580 patch_prober.go:28] interesting pod/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:27:02.867705 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:02.867342    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 16 14:27:03.867563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:03.867527    2580 patch_prober.go:28] interesting pod/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:27:03.868037 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:03.867598    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 16 14:27:04.867243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:04.867208    2580 patch_prober.go:28] interesting pod/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:27:04.867430 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:04.867265    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 16 14:27:05.394658 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:05.394624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:05.566183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:05.566128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:05.866856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:05.866821    2580 patch_prober.go:28] interesting pod/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:27:05.867025 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:05.866888    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 16 14:27:06.871095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:06.871063    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:27:06.871574 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:06.871352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:27:06.872044 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:06.872027    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:27:07.175339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:07.175252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:08.313247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.313214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:08.415990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.415951    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:08.720435 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.720404    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"]
Sep 16 14:27:08.724058 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.724042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"
Sep 16 14:27:08.727311 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.727283    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:27:08.727519 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.727504    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:27:08.737779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.737756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"]
Sep 16 14:27:08.763004 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.762975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d28f2029-90aa-4e84-bc73-272ecb3485cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dbb84810-zz78t\" (UID: \"d28f2029-90aa-4e84-bc73-272ecb3485cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"
Sep 16 14:27:08.863781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.863741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d28f2029-90aa-4e84-bc73-272ecb3485cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dbb84810-zz78t\" (UID: \"d28f2029-90aa-4e84-bc73-272ecb3485cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"
Sep 16 14:27:08.864414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:08.864393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d28f2029-90aa-4e84-bc73-272ecb3485cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dbb84810-zz78t\" (UID: \"d28f2029-90aa-4e84-bc73-272ecb3485cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"
Sep 16 14:27:09.034457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:09.034388    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"
Sep 16 14:27:09.326183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:09.326158    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"]
Sep 16 14:27:09.328633 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:27:09.328604    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd28f2029_90aa_4e84_bc73_272ecb3485cf.slice/crio-3b0167a00733d3e85f0ec7861aa985ab84130ca0da483075ea17c2395c001c6d WatchSource:0}: Error finding container 3b0167a00733d3e85f0ec7861aa985ab84130ca0da483075ea17c2395c001c6d: Status 404 returned error can't find the container with id 3b0167a00733d3e85f0ec7861aa985ab84130ca0da483075ea17c2395c001c6d
Sep 16 14:27:09.816262 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:09.816228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:09.928627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:09.928591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t" event={"ID":"d28f2029-90aa-4e84-bc73-272ecb3485cf","Type":"ContainerStarted","Data":"c26437557c6add19c53b09cdf012732d5794b2051da172319ae091ac5c7d4e92"}
Sep 16 14:27:09.928813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:09.928649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t" event={"ID":"d28f2029-90aa-4e84-bc73-272ecb3485cf","Type":"ContainerStarted","Data":"3b0167a00733d3e85f0ec7861aa985ab84130ca0da483075ea17c2395c001c6d"}
Sep 16 14:27:09.957932 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:09.957888    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t" podStartSLOduration=1.957874834 podStartE2EDuration="1.957874834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:09.956575377 +0000 UTC m=+1360.444585466" watchObservedRunningTime="2026-09-16 14:27:09.957874834 +0000 UTC m=+1360.445884910"
Sep 16 14:27:10.935543 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:10.935511    2580 generic.go:356] "Generic (PLEG): container finished" podID="d28f2029-90aa-4e84-bc73-272ecb3485cf" containerID="c26437557c6add19c53b09cdf012732d5794b2051da172319ae091ac5c7d4e92" exitCode=0
Sep 16 14:27:10.935925 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:10.935576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t" event={"ID":"d28f2029-90aa-4e84-bc73-272ecb3485cf","Type":"ContainerDied","Data":"c26437557c6add19c53b09cdf012732d5794b2051da172319ae091ac5c7d4e92"}
Sep 16 14:27:11.119159 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:11.119102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:12.069037 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.069017    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"
Sep 16 14:27:12.085778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.085756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d28f2029-90aa-4e84-bc73-272ecb3485cf-maas-api-service-ca\") pod \"d28f2029-90aa-4e84-bc73-272ecb3485cf\" (UID: \"d28f2029-90aa-4e84-bc73-272ecb3485cf\") "
Sep 16 14:27:12.086051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.086031    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d28f2029-90aa-4e84-bc73-272ecb3485cf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d28f2029-90aa-4e84-bc73-272ecb3485cf" (UID: "d28f2029-90aa-4e84-bc73-272ecb3485cf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:12.186757 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.186730    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d28f2029-90aa-4e84-bc73-272ecb3485cf-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:12.459082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.459049    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:12.947390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.947298    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"
Sep 16 14:27:12.947390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.947293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t" event={"ID":"d28f2029-90aa-4e84-bc73-272ecb3485cf","Type":"ContainerDied","Data":"3b0167a00733d3e85f0ec7861aa985ab84130ca0da483075ea17c2395c001c6d"}
Sep 16 14:27:12.947691 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:12.947442    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b0167a00733d3e85f0ec7861aa985ab84130ca0da483075ea17c2395c001c6d"
Sep 16 14:27:16.000069 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:16.000040    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:27:16.173998 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:16.173966    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:27:19.920931 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:19.920892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:20.008643 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.008613    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"]
Sep 16 14:27:20.008906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.008866    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg" podUID="6ff2b4c8-d298-4165-a639-253aafbc6abc" containerName="payload-pre-processing" containerID="cri-o://4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4" gracePeriod=30
Sep 16 14:27:20.080494 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.080463    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:20.179607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.179566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:20.270767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.270736    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:27:20.345722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.345687    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"]
Sep 16 14:27:20.350888 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.350865    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dbb84810-zz78t"]
Sep 16 14:27:20.361315 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.361287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:20.364795 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.364778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r9lm\" (UniqueName: \"kubernetes.io/projected/6ff2b4c8-d298-4165-a639-253aafbc6abc-kube-api-access-4r9lm\") pod \"6ff2b4c8-d298-4165-a639-253aafbc6abc\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") "
Sep 16 14:27:20.364883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.364851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff2b4c8-d298-4165-a639-253aafbc6abc-plugins-config-volume\") pod \"6ff2b4c8-d298-4165-a639-253aafbc6abc\" (UID: \"6ff2b4c8-d298-4165-a639-253aafbc6abc\") "
Sep 16 14:27:20.365246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.365225    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff2b4c8-d298-4165-a639-253aafbc6abc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ff2b4c8-d298-4165-a639-253aafbc6abc" (UID: "6ff2b4c8-d298-4165-a639-253aafbc6abc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:20.366818 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.366799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff2b4c8-d298-4165-a639-253aafbc6abc-kube-api-access-4r9lm" (OuterVolumeSpecName: "kube-api-access-4r9lm") pod "6ff2b4c8-d298-4165-a639-253aafbc6abc" (UID: "6ff2b4c8-d298-4165-a639-253aafbc6abc"). InnerVolumeSpecName "kube-api-access-4r9lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:20.465656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.465627    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r9lm\" (UniqueName: \"kubernetes.io/projected/6ff2b4c8-d298-4165-a639-253aafbc6abc-kube-api-access-4r9lm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.465656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.465653    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff2b4c8-d298-4165-a639-253aafbc6abc-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.479793 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.479765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:20.740160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.740051    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:20.997128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.997045    2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff2b4c8-d298-4165-a639-253aafbc6abc" containerID="4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4" exitCode=0
Sep 16 14:27:20.997128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.997090    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"
Sep 16 14:27:20.997128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.997109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg" event={"ID":"6ff2b4c8-d298-4165-a639-253aafbc6abc","Type":"ContainerDied","Data":"4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4"}
Sep 16 14:27:20.997655 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.997182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg" event={"ID":"6ff2b4c8-d298-4165-a639-253aafbc6abc","Type":"ContainerDied","Data":"fc991d423bcca64d53b23f593c3b6264fc43065ddcaf02c4b63c001417c3c6ac"}
Sep 16 14:27:20.997655 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:20.997207    2580 scope.go:117] "RemoveContainer" containerID="4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4"
Sep 16 14:27:21.007022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:21.006994    2580 scope.go:117] "RemoveContainer" containerID="4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4"
Sep 16 14:27:21.007430 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:27:21.007403    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4\": container with ID starting with 4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4 not found: ID does not exist" containerID="4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4"
Sep 16 14:27:21.007514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:21.007441    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4"} err="failed to get container status \"4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4\": rpc error: code = NotFound desc = could not find container \"4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4\": container with ID starting with 4464132dd5c82ff2d7557db2f6053e35362483412dcc4fcc482c63c19e20dad4 not found: ID does not exist"
Sep 16 14:27:21.035736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:21.035705    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"]
Sep 16 14:27:21.040005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:21.039981    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dbb84810-65f6976f78-5bmtg"]
Sep 16 14:27:21.102308 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:21.102280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:22.006418 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:22.006382    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff2b4c8-d298-4165-a639-253aafbc6abc" path="/var/lib/kubelet/pods/6ff2b4c8-d298-4165-a639-253aafbc6abc/volumes"
Sep 16 14:27:22.006791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:22.006706    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d28f2029-90aa-4e84-bc73-272ecb3485cf" path="/var/lib/kubelet/pods/d28f2029-90aa-4e84-bc73-272ecb3485cf/volumes"
Sep 16 14:27:22.680365 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:22.680322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:29.851317 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:29.851274    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:30.115471 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:30.115371    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"]
Sep 16 14:27:30.115693 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:30.115654    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy" containerID="cri-o://42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e" gracePeriod=30
Sep 16 14:27:30.754168 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:30.754119    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:31.000199 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:31.000168    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:27:31.098130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:31.098048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:33.324576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:33.324541    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:27:35.362324 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.362296    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:27:35.512079 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-certs\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512093    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71458779-7755-46c9-887b-0e63ababad22-istio-podinfo\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512115    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-socket\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512164    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71458779-7755-46c9-887b-0e63ababad22-istiod-ca-cert\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512185    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-istio-token\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512218    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-credential-socket\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512286 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512281    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-envoy\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-data\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.512597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512397    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvzfs\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-kube-api-access-vvzfs\") pod \"71458779-7755-46c9-887b-0e63ababad22\" (UID: \"71458779-7755-46c9-887b-0e63ababad22\") "
Sep 16 14:27:35.512597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512576    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.512802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512667    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71458779-7755-46c9-887b-0e63ababad22-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:35.512802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-data" (OuterVolumeSpecName: "istio-data") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.512910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512832    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.512910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512863    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.512910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512886    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71458779-7755-46c9-887b-0e63ababad22-istiod-ca-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.512910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512899    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-credential-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.513111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.512912    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-data\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.514484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.514454    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.514601 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.514539    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71458779-7755-46c9-887b-0e63ababad22-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:27:35.514798 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.514769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-istio-token" (OuterVolumeSpecName: "istio-token") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:35.514876 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.514851    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-kube-api-access-vvzfs" (OuterVolumeSpecName: "kube-api-access-vvzfs") pod "71458779-7755-46c9-887b-0e63ababad22" (UID: "71458779-7755-46c9-887b-0e63ababad22"). InnerVolumeSpecName "kube-api-access-vvzfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:35.614178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.614114    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71458779-7755-46c9-887b-0e63ababad22-istio-podinfo\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.614178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.614175    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-workload-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.614178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.614185    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-istio-token\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.614178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.614194    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71458779-7755-46c9-887b-0e63ababad22-istio-envoy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.614454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:35.614202    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvzfs\" (UniqueName: \"kubernetes.io/projected/71458779-7755-46c9-887b-0e63ababad22-kube-api-access-vvzfs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:27:36.079338 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.079306    2580 generic.go:356] "Generic (PLEG): container finished" podID="71458779-7755-46c9-887b-0e63ababad22" containerID="42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e" exitCode=0
Sep 16 14:27:36.079502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.079355    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"
Sep 16 14:27:36.079502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.079361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" event={"ID":"71458779-7755-46c9-887b-0e63ababad22","Type":"ContainerDied","Data":"42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e"}
Sep 16 14:27:36.079502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.079466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq" event={"ID":"71458779-7755-46c9-887b-0e63ababad22","Type":"ContainerDied","Data":"4a6b9bfad52dd6f30281453fefef28207d6acdf140603a06f12da781bd1c5cad"}
Sep 16 14:27:36.079502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.079492    2580 scope.go:117] "RemoveContainer" containerID="42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e"
Sep 16 14:27:36.088329 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.088310    2580 scope.go:117] "RemoveContainer" containerID="84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4"
Sep 16 14:27:36.096087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.096073    2580 scope.go:117] "RemoveContainer" containerID="42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e"
Sep 16 14:27:36.096398 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:27:36.096374    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e\": container with ID starting with 42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e not found: ID does not exist" containerID="42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e"
Sep 16 14:27:36.096514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.096408    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e"} err="failed to get container status \"42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e\": rpc error: code = NotFound desc = could not find container \"42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e\": container with ID starting with 42fdf2b44abf3b79eb60d81763b8e6fc251257908ed3de31233c1c98684b086e not found: ID does not exist"
Sep 16 14:27:36.096514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.096433    2580 scope.go:117] "RemoveContainer" containerID="84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4"
Sep 16 14:27:36.096744 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:27:36.096728    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4\": container with ID starting with 84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4 not found: ID does not exist" containerID="84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4"
Sep 16 14:27:36.096783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.096750    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4"} err="failed to get container status \"84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4\": rpc error: code = NotFound desc = could not find container \"84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4\": container with ID starting with 84b9a42fbc8402265a4374eb205e66a739b276cf872246d9ac9f30c522fb15e4 not found: ID does not exist"
Sep 16 14:27:36.114599 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.114574    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"]
Sep 16 14:27:36.120428 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:36.120401    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dbb84810-openshift-default-688fdb76cb-rctxq"]
Sep 16 14:27:38.011795 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:38.007540    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71458779-7755-46c9-887b-0e63ababad22" path="/var/lib/kubelet/pods/71458779-7755-46c9-887b-0e63ababad22/volumes"
Sep 16 14:27:38.627557 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:38.627525    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:38.834671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:38.834636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:45.381437 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:45.381405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:27:45.642091 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:45.641999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:45.755514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:45.755478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:46.825455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:46.825419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:47.169193 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169100    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"]
Sep 16 14:27:47.169525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169511    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy"
Sep 16 14:27:47.169567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169527    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy"
Sep 16 14:27:47.169567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169536    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d28f2029-90aa-4e84-bc73-272ecb3485cf" containerName="revoke-keys"
Sep 16 14:27:47.169567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169542    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d28f2029-90aa-4e84-bc73-272ecb3485cf" containerName="revoke-keys"
Sep 16 14:27:47.169567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169552    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff2b4c8-d298-4165-a639-253aafbc6abc" containerName="payload-pre-processing"
Sep 16 14:27:47.169567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169557    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff2b4c8-d298-4165-a639-253aafbc6abc" containerName="payload-pre-processing"
Sep 16 14:27:47.169567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169567    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy"
Sep 16 14:27:47.169794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169572    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy"
Sep 16 14:27:47.169794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169642    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d28f2029-90aa-4e84-bc73-272ecb3485cf" containerName="revoke-keys"
Sep 16 14:27:47.169794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169652    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff2b4c8-d298-4165-a639-253aafbc6abc" containerName="payload-pre-processing"
Sep 16 14:27:47.169794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169659    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy"
Sep 16 14:27:47.169794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.169665    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="71458779-7755-46c9-887b-0e63ababad22" containerName="istio-proxy"
Sep 16 14:27:47.174242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.174224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.177861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.177832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-2f9be3-openshift-default-dockercfg-qxjs4\""
Sep 16 14:27:47.177993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.177897    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 14:27:47.215125 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.215098    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"]
Sep 16 14:27:47.298969 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.298935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:47.312627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-envoy\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-credential-socket\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-data\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4hrk\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-kube-api-access-s4hrk\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-podinfo\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-certs\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-socket\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-token\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.312987 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.312865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istiod-ca-cert\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.413734 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.413704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-token\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.413883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.413865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istiod-ca-cert\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.413936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.413923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-envoy\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.413988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.413958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-credential-socket\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.413988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.413981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-data\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4hrk\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-kube-api-access-s4hrk\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-podinfo\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-certs\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-socket\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414374 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-data\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-credential-socket\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-socket\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istiod-ca-cert\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.414781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.414762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-certs\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.416183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.416166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-envoy\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.416390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.416371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-podinfo\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.456832 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.456774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-token\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.505044 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.505025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4hrk\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-kube-api-access-s4hrk\") pod \"e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") " pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:47.543546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.543518    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:47.784503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:47.784475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:48.092638 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:48.092615    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"]
Sep 16 14:27:48.093245 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:27:48.093222    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6014b7_e2dd_4f53_8f4e_af84cb3e894b.slice/crio-1bd59afad407e206df26adfd6994b87526e9838d51d9a65e545f7cdda3d891c7 WatchSource:0}: Error finding container 1bd59afad407e206df26adfd6994b87526e9838d51d9a65e545f7cdda3d891c7: Status 404 returned error can't find the container with id 1bd59afad407e206df26adfd6994b87526e9838d51d9a65e545f7cdda3d891c7
Sep 16 14:27:48.095374 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:48.095344    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:27:48.095447 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:48.095408    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:27:48.095447 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:48.095436    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:27:48.141922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:48.141897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" event={"ID":"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b","Type":"ContainerStarted","Data":"1bd59afad407e206df26adfd6994b87526e9838d51d9a65e545f7cdda3d891c7"}
Sep 16 14:27:49.149352 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:49.149319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" event={"ID":"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b","Type":"ContainerStarted","Data":"31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d"}
Sep 16 14:27:49.196489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:49.196444    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" podStartSLOduration=2.196428172 podStartE2EDuration="2.196428172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:49.194401331 +0000 UTC m=+1399.682411406" watchObservedRunningTime="2026-09-16 14:27:49.196428172 +0000 UTC m=+1399.684438247"
Sep 16 14:27:49.785070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:49.785039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:49.789928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:49.789905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:50.154229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:50.154130    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:50.155091 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:50.155072    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:27:50.788265 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:50.788234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:51.572420 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:51.572391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:51.731844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:51.731805    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:52.437489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:52.437451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:54.073983 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:54.073943    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:54.200476 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:54.200438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:54.893790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:54.893753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:55.004890 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.004854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:55.050969 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.050941    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"]
Sep 16 14:27:55.054457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.054441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.068048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.068030    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-2f9be3\""
Sep 16 14:27:55.082399 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.082376    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"]
Sep 16 14:27:55.084542 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.084517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41555e7-54df-4c04-bd6a-975d5cfda52d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.084655 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.084578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v5b4\" (UniqueName: \"kubernetes.io/projected/c41555e7-54df-4c04-bd6a-975d5cfda52d-kube-api-access-9v5b4\") pod \"payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.185990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.185912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41555e7-54df-4c04-bd6a-975d5cfda52d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.185990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.185950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v5b4\" (UniqueName: \"kubernetes.io/projected/c41555e7-54df-4c04-bd6a-975d5cfda52d-kube-api-access-9v5b4\") pod \"payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.186579 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.186562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41555e7-54df-4c04-bd6a-975d5cfda52d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.195529 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.195503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v5b4\" (UniqueName: \"kubernetes.io/projected/c41555e7-54df-4c04-bd6a-975d5cfda52d-kube-api-access-9v5b4\") pod \"payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.364653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.364625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:55.414464 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.414434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:55.636158 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:55.636098    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"]
Sep 16 14:27:55.638514 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:27:55.638481    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41555e7_54df_4c04_bd6a_975d5cfda52d.slice/crio-02f32c15ef6b874e740f19b7f3b5b18906815b091b909f66b1c4a08fe01ba010 WatchSource:0}: Error finding container 02f32c15ef6b874e740f19b7f3b5b18906815b091b909f66b1c4a08fe01ba010: Status 404 returned error can't find the container with id 02f32c15ef6b874e740f19b7f3b5b18906815b091b909f66b1c4a08fe01ba010
Sep 16 14:27:56.185046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:56.185013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl" event={"ID":"c41555e7-54df-4c04-bd6a-975d5cfda52d","Type":"ContainerStarted","Data":"67d724299be3d2fa6d36ecf4a69e67d78187ea585ca5aa48bb3ec6ef9bb6dd16"}
Sep 16 14:27:56.185462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:56.185059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl" event={"ID":"c41555e7-54df-4c04-bd6a-975d5cfda52d","Type":"ContainerStarted","Data":"02f32c15ef6b874e740f19b7f3b5b18906815b091b909f66b1c4a08fe01ba010"}
Sep 16 14:27:56.185462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:56.185169    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:27:56.221776 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:56.221735    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl" podStartSLOduration=1.221722303 podStartE2EDuration="1.221722303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:56.221393691 +0000 UTC m=+1406.709403780" watchObservedRunningTime="2026-09-16 14:27:56.221722303 +0000 UTC m=+1406.709732378"
Sep 16 14:27:56.636326 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:56.636296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:27:58.590133 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:27:58.590100    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:28:05.976775 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:05.976747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:07.192517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:07.192489    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:28:13.878370 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:13.878336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:28:16.903056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:16.903021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:18.529995 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:18.529956    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:19.383228 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:19.383187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:21.483236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:21.483203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:21.850716 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:21.850644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:22.251106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:22.251070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:23.135035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:23.134997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:25.350713 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.350673    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerID="31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d" exitCode=0
Sep 16 14:28:25.351207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.350753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" event={"ID":"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b","Type":"ContainerDied","Data":"31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d"}
Sep 16 14:28:25.351283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.351242    2580 scope.go:117] "RemoveContainer" containerID="31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d"
Sep 16 14:28:25.352323 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.352294    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:25.352411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.352385    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:25.352450 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.352425    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:25.816685 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.816645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:25.951280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:25.951248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:26.358080 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:26.358042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" event={"ID":"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b","Type":"ContainerStarted","Data":"0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6"}
Sep 16 14:28:26.785292 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:26.785261    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:28:26.785512 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:26.785490    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:28:26.785570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:26.785550    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 16 14:28:26.983426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:26.983391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:27.275073 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:27.275040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:28:27.785194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:27.785156    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:28:27.785666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:27.785214    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 16 14:28:28.024160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.022258    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:28.111957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.111867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:28.324366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.324328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:28.745618 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.745589    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"]
Sep 16 14:28:28.748300 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.748284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:28.754480 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.754457    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-cbeb11-dockercfg-twskv\""
Sep 16 14:28:28.756111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.756095    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-cbeb11\""
Sep 16 14:28:28.772684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.772658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"]
Sep 16 14:28:28.785435 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.785413    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:28:28.785704 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.785460    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 16 14:28:28.794903 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.794881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2tsg\" (UniqueName: \"kubernetes.io/projected/470719d5-e9d5-4815-8cce-344097d1b877-kube-api-access-h2tsg\") pod \"payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:28.794992 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.794924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/470719d5-e9d5-4815-8cce-344097d1b877-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:28.825694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.825663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:28.895490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.895468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/470719d5-e9d5-4815-8cce-344097d1b877-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:28.895606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.895574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2tsg\" (UniqueName: \"kubernetes.io/projected/470719d5-e9d5-4815-8cce-344097d1b877-kube-api-access-h2tsg\") pod \"payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:28.896101 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.896083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/470719d5-e9d5-4815-8cce-344097d1b877-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:28.922878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.922846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2tsg\" (UniqueName: \"kubernetes.io/projected/470719d5-e9d5-4815-8cce-344097d1b877-kube-api-access-h2tsg\") pod \"payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:28.967408 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:28.967382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:29.058962 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.058881    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:29.319847 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.319751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:29.356248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.356227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"]
Sep 16 14:28:29.358009 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:28:29.357979    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470719d5_e9d5_4815_8cce_344097d1b877.slice/crio-35cac17ec538a9bd6bd5e0adaf79a791e07c8e8ef91857f2188b673ee186fa50 WatchSource:0}: Error finding container 35cac17ec538a9bd6bd5e0adaf79a791e07c8e8ef91857f2188b673ee186fa50: Status 404 returned error can't find the container with id 35cac17ec538a9bd6bd5e0adaf79a791e07c8e8ef91857f2188b673ee186fa50
Sep 16 14:28:29.375093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.375070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7" event={"ID":"470719d5-e9d5-4815-8cce-344097d1b877","Type":"ContainerStarted","Data":"35cac17ec538a9bd6bd5e0adaf79a791e07c8e8ef91857f2188b673ee186fa50"}
Sep 16 14:28:29.435972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.435946    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8959-69a974"]
Sep 16 14:28:29.438377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.438360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:29.442197 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.442177    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8t2sn\""
Sep 16 14:28:29.450596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.450574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8959-69a974"]
Sep 16 14:28:29.524001 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.523958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn46r\" (UniqueName: \"kubernetes.io/projected/5887b90c-fef2-4acd-838b-52e3300bfcf7-kube-api-access-dn46r\") pod \"test-curl-8959-69a974\" (UID: \"5887b90c-fef2-4acd-838b-52e3300bfcf7\") " pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:29.624747 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.624658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn46r\" (UniqueName: \"kubernetes.io/projected/5887b90c-fef2-4acd-838b-52e3300bfcf7-kube-api-access-dn46r\") pod \"test-curl-8959-69a974\" (UID: \"5887b90c-fef2-4acd-838b-52e3300bfcf7\") " pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:29.633506 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.633481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn46r\" (UniqueName: \"kubernetes.io/projected/5887b90c-fef2-4acd-838b-52e3300bfcf7-kube-api-access-dn46r\") pod \"test-curl-8959-69a974\" (UID: \"5887b90c-fef2-4acd-838b-52e3300bfcf7\") " pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:29.747159 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.747102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:29.785805 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.785769    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:28:29.786207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:29.785828    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 16 14:28:30.050953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.050930    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8959-69a974"]
Sep 16 14:28:30.052337 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:28:30.052290    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5887b90c_fef2_4acd_838b_52e3300bfcf7.slice/crio-7bbfb10fc95993a5437532cebf70f7355777747630a16cebc66a56681c05db2a WatchSource:0}: Error finding container 7bbfb10fc95993a5437532cebf70f7355777747630a16cebc66a56681c05db2a: Status 404 returned error can't find the container with id 7bbfb10fc95993a5437532cebf70f7355777747630a16cebc66a56681c05db2a
Sep 16 14:28:30.338443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.338351    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:30.381353 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.381317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7" event={"ID":"470719d5-e9d5-4815-8cce-344097d1b877","Type":"ContainerStarted","Data":"ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32"}
Sep 16 14:28:30.381553 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.381441    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:30.382909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.382885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8959-69a974" event={"ID":"5887b90c-fef2-4acd-838b-52e3300bfcf7","Type":"ContainerStarted","Data":"061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396"}
Sep 16 14:28:30.383028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.382918    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8959-69a974" event={"ID":"5887b90c-fef2-4acd-838b-52e3300bfcf7","Type":"ContainerStarted","Data":"7bbfb10fc95993a5437532cebf70f7355777747630a16cebc66a56681c05db2a"}
Sep 16 14:28:30.404414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.403955    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7" podStartSLOduration=2.403935569 podStartE2EDuration="2.403935569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:30.400574743 +0000 UTC m=+1440.888584819" watchObservedRunningTime="2026-09-16 14:28:30.403935569 +0000 UTC m=+1440.891945645"
Sep 16 14:28:30.421249 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.421171    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8959-69a974" podStartSLOduration=1.421122411 podStartE2EDuration="1.421122411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:30.418434679 +0000 UTC m=+1440.906444756" watchObservedRunningTime="2026-09-16 14:28:30.421122411 +0000 UTC m=+1440.909132490"
Sep 16 14:28:30.785045 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.785013    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:28:30.785243 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:30.785065    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 16 14:28:31.095280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:31.095183    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8959-69a974"]
Sep 16 14:28:31.100985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:31.100960    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8959-69a974"]
Sep 16 14:28:31.162327 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:31.162290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:31.388171 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:31.388073    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8959-69a974" container="test-curl-8959-69a974"
Sep 16 14:28:31.788861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:31.788830    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:28:31.789112 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:31.789095    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:28:31.789840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:31.789822    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:28:32.000207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:32.000175    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:28:32.393013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:32.392972    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8959-69a974" podUID="5887b90c-fef2-4acd-838b-52e3300bfcf7" containerName="test-curl-8959-69a974" containerID="cri-o://061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396" gracePeriod=2
Sep 16 14:28:32.670746 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:32.670666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:33.749517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:33.749486    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"]
Sep 16 14:28:33.773974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:33.773948    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:34.578564 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:34.578545    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:34.674887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:34.674813    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn46r\" (UniqueName: \"kubernetes.io/projected/5887b90c-fef2-4acd-838b-52e3300bfcf7-kube-api-access-dn46r\") pod \"5887b90c-fef2-4acd-838b-52e3300bfcf7\" (UID: \"5887b90c-fef2-4acd-838b-52e3300bfcf7\") "
Sep 16 14:28:34.676767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:34.676742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5887b90c-fef2-4acd-838b-52e3300bfcf7-kube-api-access-dn46r" (OuterVolumeSpecName: "kube-api-access-dn46r") pod "5887b90c-fef2-4acd-838b-52e3300bfcf7" (UID: "5887b90c-fef2-4acd-838b-52e3300bfcf7"). InnerVolumeSpecName "kube-api-access-dn46r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:34.776396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:34.776364    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn46r\" (UniqueName: \"kubernetes.io/projected/5887b90c-fef2-4acd-838b-52e3300bfcf7-kube-api-access-dn46r\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:35.410777 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:35.410744    2580 generic.go:356] "Generic (PLEG): container finished" podID="5887b90c-fef2-4acd-838b-52e3300bfcf7" containerID="061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396" exitCode=137
Sep 16 14:28:35.410956 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:35.410792    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:35.410956 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:35.410806    2580 scope.go:117] "RemoveContainer" containerID="061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396"
Sep 16 14:28:35.419591 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:35.419572    2580 scope.go:117] "RemoveContainer" containerID="061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396"
Sep 16 14:28:35.419871 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:35.419852    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396\": container with ID starting with 061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396 not found: ID does not exist" containerID="061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396"
Sep 16 14:28:35.419919 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:35.419879    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396"} err="failed to get container status \"061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396\": rpc error: code = NotFound desc = could not find container \"061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396\": container with ID starting with 061345ca49b09073e0f03315957ab61279e59b64832f4c1edce7f338de75d396 not found: ID does not exist"
Sep 16 14:28:35.427624 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:35.427598    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8959-69a974\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="5887b90c-fef2-4acd-838b-52e3300bfcf7" pod="opendatahub/test-curl-8959-69a974"
Sep 16 14:28:36.007407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:36.007372    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5887b90c-fef2-4acd-838b-52e3300bfcf7" path="/var/lib/kubelet/pods/5887b90c-fef2-4acd-838b-52e3300bfcf7/volumes"
Sep 16 14:28:41.390790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:41.390762    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:28:41.496018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:41.495986    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:28:48.768270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.768210    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg" podUID="89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" containerName="main" containerID="cri-o://612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a" gracePeriod=45
Sep 16 14:28:48.797493 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.797463    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"]
Sep 16 14:28:48.797975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.797957    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5887b90c-fef2-4acd-838b-52e3300bfcf7" containerName="test-curl-8959-69a974"
Sep 16 14:28:48.797975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.797974    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5887b90c-fef2-4acd-838b-52e3300bfcf7" containerName="test-curl-8959-69a974"
Sep 16 14:28:48.798128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.798043    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5887b90c-fef2-4acd-838b-52e3300bfcf7" containerName="test-curl-8959-69a974"
Sep 16 14:28:48.801095 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.801074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.805353 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.805329    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-50b57c\"/\"sub-test-model-50b57c-kserve-self-signed-certs\""
Sep 16 14:28:48.805458 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.805421    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-50b57c\"/\"kube-root-ca.crt\""
Sep 16 14:28:48.806006 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.805832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-50b57c\"/\"default-dockercfg-sjkdw\""
Sep 16 14:28:48.806006 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.805878    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-50b57c\"/\"openshift-service-ca.crt\""
Sep 16 14:28:48.815406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.815382    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"]
Sep 16 14:28:48.897392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.897367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-model-cache\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.897505 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.897430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-home\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.897570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.897515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tls-certs\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.897570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.897548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmrrn\" (UniqueName: \"kubernetes.io/projected/ee31a4e0-85f2-48d2-91e3-821cadb7984c-kube-api-access-vmrrn\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.897680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.897628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-dshm\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.897680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.897655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tmp-dir\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999019 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.998993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-model-cache\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999145 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-home\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999145 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tls-certs\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999145 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmrrn\" (UniqueName: \"kubernetes.io/projected/ee31a4e0-85f2-48d2-91e3-821cadb7984c-kube-api-access-vmrrn\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-dshm\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tmp-dir\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999451    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-model-cache\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-home\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:48.999635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:48.999621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tmp-dir\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:49.001273 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.001255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-dshm\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:49.001419 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.001401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tls-certs\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:49.019992 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.019943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:28:49.042386 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.042368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmrrn\" (UniqueName: \"kubernetes.io/projected/ee31a4e0-85f2-48d2-91e3-821cadb7984c-kube-api-access-vmrrn\") pod \"sub-test-model-50b57c-kserve-77bb58c787-8tmd5\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:49.099770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.099738    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tls-certs\") pod \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") "
Sep 16 14:28:49.099901 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.099829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-home\") pod \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") "
Sep 16 14:28:49.099901 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.099844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-dshm\") pod \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") "
Sep 16 14:28:49.099901 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.099862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tmp-dir\") pod \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") "
Sep 16 14:28:49.100084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.099902    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srrmr\" (UniqueName: \"kubernetes.io/projected/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-kube-api-access-srrmr\") pod \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") "
Sep 16 14:28:49.100084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.099968    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-model-cache\") pod \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\" (UID: \"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e\") "
Sep 16 14:28:49.100240 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.100094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-home" (OuterVolumeSpecName: "home") pod "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" (UID: "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:49.100240 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.100115    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" (UID: "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:49.100354 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.100290    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-model-cache" (OuterVolumeSpecName: "model-cache") pod "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" (UID: "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:49.100437 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.100417    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:49.100501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.100442    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:49.100501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.100455    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:49.102026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.101997    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-dshm" (OuterVolumeSpecName: "dshm") pod "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" (UID: "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:49.102026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.102015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" (UID: "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:28:49.102205 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.102186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-kube-api-access-srrmr" (OuterVolumeSpecName: "kube-api-access-srrmr") pod "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" (UID: "89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e"). InnerVolumeSpecName "kube-api-access-srrmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:49.125986 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.125967    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:49.202022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.201988    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srrmr\" (UniqueName: \"kubernetes.io/projected/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-kube-api-access-srrmr\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:49.202022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.202028    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:49.202231 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.202044    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:49.400681 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.400657    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"]
Sep 16 14:28:49.401362 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:28:49.401332    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee31a4e0_85f2_48d2_91e3_821cadb7984c.slice/crio-e63c693fcfe3a2fb5ec2e3a04b341871f87b4416741fa08a3978770d047cc822 WatchSource:0}: Error finding container e63c693fcfe3a2fb5ec2e3a04b341871f87b4416741fa08a3978770d047cc822: Status 404 returned error can't find the container with id e63c693fcfe3a2fb5ec2e3a04b341871f87b4416741fa08a3978770d047cc822
Sep 16 14:28:49.497572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.497545    2580 generic.go:356] "Generic (PLEG): container finished" podID="89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" containerID="612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a" exitCode=0
Sep 16 14:28:49.497694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.497582    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"
Sep 16 14:28:49.497694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.497620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg" event={"ID":"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e","Type":"ContainerDied","Data":"612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a"}
Sep 16 14:28:49.497694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.497666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg" event={"ID":"89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e","Type":"ContainerDied","Data":"fab9fa4f65304cbd61cb4374d1bef908cdb12b7c7a77ad31761cd64f20d35b00"}
Sep 16 14:28:49.497694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.497684    2580 scope.go:117] "RemoveContainer" containerID="612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a"
Sep 16 14:28:49.499528 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.499507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5" event={"ID":"ee31a4e0-85f2-48d2-91e3-821cadb7984c","Type":"ContainerStarted","Data":"7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63"}
Sep 16 14:28:49.499647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.499537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5" event={"ID":"ee31a4e0-85f2-48d2-91e3-821cadb7984c","Type":"ContainerStarted","Data":"e63c693fcfe3a2fb5ec2e3a04b341871f87b4416741fa08a3978770d047cc822"}
Sep 16 14:28:49.507279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.507260    2580 scope.go:117] "RemoveContainer" containerID="612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a"
Sep 16 14:28:49.507515 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:49.507495    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a\": container with ID starting with 612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a not found: ID does not exist" containerID="612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a"
Sep 16 14:28:49.507573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.507524    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a"} err="failed to get container status \"612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a\": rpc error: code = NotFound desc = could not find container \"612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a\": container with ID starting with 612fb0d54c8eaabf8ff904e3c5534db2deb702b36c53fff37b035d2fad7c4b9a not found: ID does not exist"
Sep 16 14:28:49.525019 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.524963    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5" podStartSLOduration=1.524950478 podStartE2EDuration="1.524950478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:49.521703757 +0000 UTC m=+1460.009713830" watchObservedRunningTime="2026-09-16 14:28:49.524950478 +0000 UTC m=+1460.012960554"
Sep 16 14:28:49.539893 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.539876    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"]
Sep 16 14:28:49.543282 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:49.543265    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/auth-test-model-f72402-kserve-58df7b887f-625zg"]
Sep 16 14:28:50.005870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:50.005836    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" path="/var/lib/kubelet/pods/89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e/volumes"
Sep 16 14:28:50.358196 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:50.358105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:50.658181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:50.658089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:50.818899 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:50.818865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:51.474868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.474840    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8956-709d4c"]
Sep 16 14:28:51.475297 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.475283    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" containerName="main"
Sep 16 14:28:51.475297 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.475298    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" containerName="main"
Sep 16 14:28:51.475376 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.475360    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e8c2f0-212e-4ba5-88d7-5ffcb53ad23e" containerName="main"
Sep 16 14:28:51.479810 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.479793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:51.482434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.482412    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8t2sn\""
Sep 16 14:28:51.483125 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.483100    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-709d4c"]
Sep 16 14:28:51.622427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.622402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npvg9\" (UniqueName: \"kubernetes.io/projected/d34bc852-3c83-4d35-a5f3-43dab66e9e2b-kube-api-access-npvg9\") pod \"test-curl-8956-709d4c\" (UID: \"d34bc852-3c83-4d35-a5f3-43dab66e9e2b\") " pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:51.694790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.694764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:51.722885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.722862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npvg9\" (UniqueName: \"kubernetes.io/projected/d34bc852-3c83-4d35-a5f3-43dab66e9e2b-kube-api-access-npvg9\") pod \"test-curl-8956-709d4c\" (UID: \"d34bc852-3c83-4d35-a5f3-43dab66e9e2b\") " pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:51.732497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.732440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npvg9\" (UniqueName: \"kubernetes.io/projected/d34bc852-3c83-4d35-a5f3-43dab66e9e2b-kube-api-access-npvg9\") pod \"test-curl-8956-709d4c\" (UID: \"d34bc852-3c83-4d35-a5f3-43dab66e9e2b\") " pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:51.790353 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:51.790330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:52.070962 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:28:52.070935    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd34bc852_3c83_4d35_a5f3_43dab66e9e2b.slice/crio-b285b4e9694da0aecedaaa7b38a575cabb5fd90bc52f5905686dbf28465ffb20 WatchSource:0}: Error finding container b285b4e9694da0aecedaaa7b38a575cabb5fd90bc52f5905686dbf28465ffb20: Status 404 returned error can't find the container with id b285b4e9694da0aecedaaa7b38a575cabb5fd90bc52f5905686dbf28465ffb20
Sep 16 14:28:52.071478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:52.071456    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-709d4c"]
Sep 16 14:28:52.517828 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:52.517803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-709d4c" event={"ID":"d34bc852-3c83-4d35-a5f3-43dab66e9e2b","Type":"ContainerStarted","Data":"c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da"}
Sep 16 14:28:52.518203 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:52.517838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-709d4c" event={"ID":"d34bc852-3c83-4d35-a5f3-43dab66e9e2b","Type":"ContainerStarted","Data":"b285b4e9694da0aecedaaa7b38a575cabb5fd90bc52f5905686dbf28465ffb20"}
Sep 16 14:28:52.535555 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:52.535422    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8956-709d4c" podStartSLOduration=1.535405818 podStartE2EDuration="1.535405818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:52.534802693 +0000 UTC m=+1463.022812769" watchObservedRunningTime="2026-09-16 14:28:52.535405818 +0000 UTC m=+1463.023415895"
Sep 16 14:28:52.675503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:52.675354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:53.194194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.194162    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8956-709d4c"]
Sep 16 14:28:53.201093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.201063    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8956-709d4c"]
Sep 16 14:28:53.336289 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.336258    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8956-75466f"]
Sep 16 14:28:53.336757 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.336738    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34bc852-3c83-4d35-a5f3-43dab66e9e2b" containerName="test-curl-8956-709d4c"
Sep 16 14:28:53.336852 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.336759    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34bc852-3c83-4d35-a5f3-43dab66e9e2b" containerName="test-curl-8956-709d4c"
Sep 16 14:28:53.336907 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.336876    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d34bc852-3c83-4d35-a5f3-43dab66e9e2b" containerName="test-curl-8956-709d4c"
Sep 16 14:28:53.339996 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.339978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:53.349474 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.349453    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-75466f"]
Sep 16 14:28:53.383625 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.383598    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:28:53.437697 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.437676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzfgv\" (UniqueName: \"kubernetes.io/projected/ea96f7de-bfff-40ad-8105-562ca214381e-kube-api-access-jzfgv\") pod \"test-curl-8956-75466f\" (UID: \"ea96f7de-bfff-40ad-8105-562ca214381e\") " pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:53.523732 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:53.523710    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8956-709d4c" container="test-curl-8956-709d4c"
Sep 16 14:28:53.538708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.538682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzfgv\" (UniqueName: \"kubernetes.io/projected/ea96f7de-bfff-40ad-8105-562ca214381e-kube-api-access-jzfgv\") pod \"test-curl-8956-75466f\" (UID: \"ea96f7de-bfff-40ad-8105-562ca214381e\") " pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:53.547645 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.547625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzfgv\" (UniqueName: \"kubernetes.io/projected/ea96f7de-bfff-40ad-8105-562ca214381e-kube-api-access-jzfgv\") pod \"test-curl-8956-75466f\" (UID: \"ea96f7de-bfff-40ad-8105-562ca214381e\") " pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:53.649193 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.649118    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:53.719639 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.719614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:28:53.917596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:53.917423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-75466f"]
Sep 16 14:28:53.920101 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:28:53.920070    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea96f7de_bfff_40ad_8105_562ca214381e.slice/crio-05415f67630a9218a0c8c9cc87181f87530be6a08f087b56f218f43ebb50b976 WatchSource:0}: Error finding container 05415f67630a9218a0c8c9cc87181f87530be6a08f087b56f218f43ebb50b976: Status 404 returned error can't find the container with id 05415f67630a9218a0c8c9cc87181f87530be6a08f087b56f218f43ebb50b976
Sep 16 14:28:54.530066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:54.530030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-75466f" event={"ID":"ea96f7de-bfff-40ad-8105-562ca214381e","Type":"ContainerStarted","Data":"20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d"}
Sep 16 14:28:54.530460 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:54.530073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-75466f" event={"ID":"ea96f7de-bfff-40ad-8105-562ca214381e","Type":"ContainerStarted","Data":"05415f67630a9218a0c8c9cc87181f87530be6a08f087b56f218f43ebb50b976"}
Sep 16 14:28:54.530460 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:54.530102    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8956-709d4c" podUID="d34bc852-3c83-4d35-a5f3-43dab66e9e2b" containerName="test-curl-8956-709d4c" containerID="cri-o://c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da" gracePeriod=2
Sep 16 14:28:54.550947 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:54.550912    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8956-75466f" podStartSLOduration=1.5508995780000001 podStartE2EDuration="1.550899578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:54.550617206 +0000 UTC m=+1465.038627281" watchObservedRunningTime="2026-09-16 14:28:54.550899578 +0000 UTC m=+1465.038909652"
Sep 16 14:28:55.000023 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:54.999999    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:28:55.207673 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.207643    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8956-75466f"]
Sep 16 14:28:55.212228 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.212205    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8956-75466f"]
Sep 16 14:28:55.328162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.328068    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8956-b79526"]
Sep 16 14:28:55.328535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.328522    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" containerName="test-curl-8956-75466f"
Sep 16 14:28:55.328578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.328537    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" containerName="test-curl-8956-75466f"
Sep 16 14:28:55.328627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.328617    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" containerName="test-curl-8956-75466f"
Sep 16 14:28:55.331800 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.331785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:28:55.342260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.341986    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-b79526"]
Sep 16 14:28:55.453676 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.453651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdgl6\" (UniqueName: \"kubernetes.io/projected/5c460f5c-a92c-4a10-b94d-f5d50bb91a49-kube-api-access-wdgl6\") pod \"test-curl-8956-b79526\" (UID: \"5c460f5c-a92c-4a10-b94d-f5d50bb91a49\") " pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:28:55.535327 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:55.535306    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8956-75466f" container="test-curl-8956-75466f"
Sep 16 14:28:55.555074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.555050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdgl6\" (UniqueName: \"kubernetes.io/projected/5c460f5c-a92c-4a10-b94d-f5d50bb91a49-kube-api-access-wdgl6\") pod \"test-curl-8956-b79526\" (UID: \"5c460f5c-a92c-4a10-b94d-f5d50bb91a49\") " pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:28:55.569512 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.569490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdgl6\" (UniqueName: \"kubernetes.io/projected/5c460f5c-a92c-4a10-b94d-f5d50bb91a49-kube-api-access-wdgl6\") pod \"test-curl-8956-b79526\" (UID: \"5c460f5c-a92c-4a10-b94d-f5d50bb91a49\") " pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:28:55.642872 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.642804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:28:55.911781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:55.911760    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-b79526"]
Sep 16 14:28:55.913783 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:28:55.913735    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c460f5c_a92c_4a10_b94d_f5d50bb91a49.slice/crio-8adb43de1a2375e1de5f25bc7013dc8e4634c95da72589c75252a176a0debf14 WatchSource:0}: Error finding container 8adb43de1a2375e1de5f25bc7013dc8e4634c95da72589c75252a176a0debf14: Status 404 returned error can't find the container with id 8adb43de1a2375e1de5f25bc7013dc8e4634c95da72589c75252a176a0debf14
Sep 16 14:28:56.541626 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.541594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-b79526" event={"ID":"5c460f5c-a92c-4a10-b94d-f5d50bb91a49","Type":"ContainerStarted","Data":"a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e"}
Sep 16 14:28:56.541972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.541639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-b79526" event={"ID":"5c460f5c-a92c-4a10-b94d-f5d50bb91a49","Type":"ContainerStarted","Data":"8adb43de1a2375e1de5f25bc7013dc8e4634c95da72589c75252a176a0debf14"}
Sep 16 14:28:56.541972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.541752    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8956-75466f" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" containerName="test-curl-8956-75466f" containerID="cri-o://20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d" gracePeriod=2
Sep 16 14:28:56.558886 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.558844    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8956-b79526" podStartSLOduration=1.558827107 podStartE2EDuration="1.558827107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:56.557028003 +0000 UTC m=+1467.045038079" watchObservedRunningTime="2026-09-16 14:28:56.558827107 +0000 UTC m=+1467.046837179"
Sep 16 14:28:56.674808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.674785    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:56.766441 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.766414    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npvg9\" (UniqueName: \"kubernetes.io/projected/d34bc852-3c83-4d35-a5f3-43dab66e9e2b-kube-api-access-npvg9\") pod \"d34bc852-3c83-4d35-a5f3-43dab66e9e2b\" (UID: \"d34bc852-3c83-4d35-a5f3-43dab66e9e2b\") "
Sep 16 14:28:56.768332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.768305    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34bc852-3c83-4d35-a5f3-43dab66e9e2b-kube-api-access-npvg9" (OuterVolumeSpecName: "kube-api-access-npvg9") pod "d34bc852-3c83-4d35-a5f3-43dab66e9e2b" (UID: "d34bc852-3c83-4d35-a5f3-43dab66e9e2b"). InnerVolumeSpecName "kube-api-access-npvg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:56.867546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:56.867490    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npvg9\" (UniqueName: \"kubernetes.io/projected/d34bc852-3c83-4d35-a5f3-43dab66e9e2b-kube-api-access-npvg9\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:57.209155 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.209114    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8956-b79526"]
Sep 16 14:28:57.216530 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.216508    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8956-b79526"]
Sep 16 14:28:57.334078 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.334052    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8956-2ba13e"]
Sep 16 14:28:57.334508 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.334494    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c460f5c-a92c-4a10-b94d-f5d50bb91a49" containerName="test-curl-8956-b79526"
Sep 16 14:28:57.334562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.334510    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c460f5c-a92c-4a10-b94d-f5d50bb91a49" containerName="test-curl-8956-b79526"
Sep 16 14:28:57.334595 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.334586    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c460f5c-a92c-4a10-b94d-f5d50bb91a49" containerName="test-curl-8956-b79526"
Sep 16 14:28:57.337605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.337586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:28:57.347176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.347150    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-2ba13e"]
Sep 16 14:28:57.473062 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.473006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmfd6\" (UniqueName: \"kubernetes.io/projected/ca2a4982-7385-43c8-8b4a-7c4e6eca807b-kube-api-access-dmfd6\") pod \"test-curl-8956-2ba13e\" (UID: \"ca2a4982-7385-43c8-8b4a-7c4e6eca807b\") " pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:28:57.548360 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.548331    2580 generic.go:356] "Generic (PLEG): container finished" podID="d34bc852-3c83-4d35-a5f3-43dab66e9e2b" containerID="c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da" exitCode=137
Sep 16 14:28:57.548672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.548375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:57.548672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.548417    2580 scope.go:117] "RemoveContainer" containerID="c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da"
Sep 16 14:28:57.548779 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:57.548687    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8956-b79526" container="test-curl-8956-b79526"
Sep 16 14:28:57.557765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.557749    2580 scope.go:117] "RemoveContainer" containerID="c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da"
Sep 16 14:28:57.558005 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:57.557988    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da\": container with ID starting with c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da not found: ID does not exist" containerID="c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da"
Sep 16 14:28:57.558063 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.558012    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da"} err="failed to get container status \"c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da\": rpc error: code = NotFound desc = could not find container \"c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da\": container with ID starting with c9f8211ac22926ffd9e10fb1144ac9f96bc402e9f818d0b8b2114c4d9bb2d9da not found: ID does not exist"
Sep 16 14:28:57.559166 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:57.559122    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8956-709d4c\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="d34bc852-3c83-4d35-a5f3-43dab66e9e2b" pod="opendatahub/test-curl-8956-709d4c"
Sep 16 14:28:57.573531 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.573507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmfd6\" (UniqueName: \"kubernetes.io/projected/ca2a4982-7385-43c8-8b4a-7c4e6eca807b-kube-api-access-dmfd6\") pod \"test-curl-8956-2ba13e\" (UID: \"ca2a4982-7385-43c8-8b4a-7c4e6eca807b\") " pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:28:57.582765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.582741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmfd6\" (UniqueName: \"kubernetes.io/projected/ca2a4982-7385-43c8-8b4a-7c4e6eca807b-kube-api-access-dmfd6\") pod \"test-curl-8956-2ba13e\" (UID: \"ca2a4982-7385-43c8-8b4a-7c4e6eca807b\") " pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:28:57.647693 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.647673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:28:57.916005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:57.915931    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8956-2ba13e"]
Sep 16 14:28:57.918885 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:28:57.918856    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca2a4982_7385_43c8_8b4a_7c4e6eca807b.slice/crio-885fb487e2a5c5b25cdd9f74c5617741d578844a13420e5dd333090f3a8302e4 WatchSource:0}: Error finding container 885fb487e2a5c5b25cdd9f74c5617741d578844a13420e5dd333090f3a8302e4: Status 404 returned error can't find the container with id 885fb487e2a5c5b25cdd9f74c5617741d578844a13420e5dd333090f3a8302e4
Sep 16 14:28:58.006030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.005995    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d34bc852-3c83-4d35-a5f3-43dab66e9e2b" path="/var/lib/kubelet/pods/d34bc852-3c83-4d35-a5f3-43dab66e9e2b/volumes"
Sep 16 14:28:58.556076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.556050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-2ba13e" event={"ID":"ca2a4982-7385-43c8-8b4a-7c4e6eca807b","Type":"ContainerStarted","Data":"4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa"}
Sep 16 14:28:58.556395 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.556090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8956-2ba13e" event={"ID":"ca2a4982-7385-43c8-8b4a-7c4e6eca807b","Type":"ContainerStarted","Data":"885fb487e2a5c5b25cdd9f74c5617741d578844a13420e5dd333090f3a8302e4"}
Sep 16 14:28:58.556395 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.556194    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8956-b79526" podUID="5c460f5c-a92c-4a10-b94d-f5d50bb91a49" containerName="test-curl-8956-b79526" containerID="cri-o://a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e" gracePeriod=2
Sep 16 14:28:58.591094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.591054    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8956-2ba13e" podStartSLOduration=1.591039292 podStartE2EDuration="1.591039292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:58.590289378 +0000 UTC m=+1469.078299479" watchObservedRunningTime="2026-09-16 14:28:58.591039292 +0000 UTC m=+1469.079049368"
Sep 16 14:28:58.676623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.676602    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:58.782201 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.782173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzfgv\" (UniqueName: \"kubernetes.io/projected/ea96f7de-bfff-40ad-8105-562ca214381e-kube-api-access-jzfgv\") pod \"ea96f7de-bfff-40ad-8105-562ca214381e\" (UID: \"ea96f7de-bfff-40ad-8105-562ca214381e\") "
Sep 16 14:28:58.784094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.784073    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea96f7de-bfff-40ad-8105-562ca214381e-kube-api-access-jzfgv" (OuterVolumeSpecName: "kube-api-access-jzfgv") pod "ea96f7de-bfff-40ad-8105-562ca214381e" (UID: "ea96f7de-bfff-40ad-8105-562ca214381e"). InnerVolumeSpecName "kube-api-access-jzfgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:58.883182 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:58.883100    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzfgv\" (UniqueName: \"kubernetes.io/projected/ea96f7de-bfff-40ad-8105-562ca214381e-kube-api-access-jzfgv\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:28:59.126778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.126748    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:59.126914 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.126795    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:59.138449 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.138392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:59.218269 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.218244    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8956-2ba13e"]
Sep 16 14:28:59.222852 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.222830    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8956-2ba13e"]
Sep 16 14:28:59.563717 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.563682    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea96f7de-bfff-40ad-8105-562ca214381e" containerID="20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d" exitCode=137
Sep 16 14:28:59.564090 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.563740    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:59.564090 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.563784    2580 scope.go:117] "RemoveContainer" containerID="20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d"
Sep 16 14:28:59.564400 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:59.564366    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8956-2ba13e" container="test-curl-8956-2ba13e"
Sep 16 14:28:59.574499 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.574477    2580 scope.go:117] "RemoveContainer" containerID="20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d"
Sep 16 14:28:59.574602 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:59.574506    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8956-75466f\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:59.574797 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:59.574776    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d\": container with ID starting with 20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d not found: ID does not exist" containerID="20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d"
Sep 16 14:28:59.574855 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.574804    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d"} err="failed to get container status \"20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d\": rpc error: code = NotFound desc = could not find container \"20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d\": container with ID starting with 20f82384fdcb7d13c4e47b72f95cde1ef6dcbbe126dde0e56f50785616d58e0d not found: ID does not exist"
Sep 16 14:28:59.575669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.575651    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:28:59.600264 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:28:59.600237    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8956-75466f\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:28:59.829406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:28:59.829330    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:00.004411 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:29:00.004376    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8956-75466f\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" pod="opendatahub/test-curl-8956-75466f"
Sep 16 14:29:00.006150 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.006116    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea96f7de-bfff-40ad-8105-562ca214381e" path="/var/lib/kubelet/pods/ea96f7de-bfff-40ad-8105-562ca214381e/volumes"
Sep 16 14:29:00.149803 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.149720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:00.530761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.530728    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:00.571126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.571099    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8956-2ba13e" podUID="ca2a4982-7385-43c8-8b4a-7c4e6eca807b" containerName="test-curl-8956-2ba13e" containerID="cri-o://4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa" gracePeriod=2
Sep 16 14:29:00.729982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.729960    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:29:00.799725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.799663    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdgl6\" (UniqueName: \"kubernetes.io/projected/5c460f5c-a92c-4a10-b94d-f5d50bb91a49-kube-api-access-wdgl6\") pod \"5c460f5c-a92c-4a10-b94d-f5d50bb91a49\" (UID: \"5c460f5c-a92c-4a10-b94d-f5d50bb91a49\") "
Sep 16 14:29:00.801575 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.801552    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c460f5c-a92c-4a10-b94d-f5d50bb91a49-kube-api-access-wdgl6" (OuterVolumeSpecName: "kube-api-access-wdgl6") pod "5c460f5c-a92c-4a10-b94d-f5d50bb91a49" (UID: "5c460f5c-a92c-4a10-b94d-f5d50bb91a49"). InnerVolumeSpecName "kube-api-access-wdgl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:29:00.901049 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:00.901030    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdgl6\" (UniqueName: \"kubernetes.io/projected/5c460f5c-a92c-4a10-b94d-f5d50bb91a49-kube-api-access-wdgl6\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:29:01.578487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:01.578444    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c460f5c-a92c-4a10-b94d-f5d50bb91a49" containerID="a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e" exitCode=137
Sep 16 14:29:01.578916 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:01.578496    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:29:01.578916 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:01.578540    2580 scope.go:117] "RemoveContainer" containerID="a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e"
Sep 16 14:29:01.588328 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:01.588310    2580 scope.go:117] "RemoveContainer" containerID="a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e"
Sep 16 14:29:01.588627 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:29:01.588609    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e\": container with ID starting with a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e not found: ID does not exist" containerID="a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e"
Sep 16 14:29:01.588689 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:01.588637    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e"} err="failed to get container status \"a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e\": rpc error: code = NotFound desc = could not find container \"a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e\": container with ID starting with a6667f1736e1b5e6aa78dd3ab0414e1e8ea16a35c9571ac9e5436220ebc2998e not found: ID does not exist"
Sep 16 14:29:01.592454 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:29:01.592431    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8956-b79526\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="5c460f5c-a92c-4a10-b94d-f5d50bb91a49" pod="opendatahub/test-curl-8956-b79526"
Sep 16 14:29:02.005945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:02.005914    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c460f5c-a92c-4a10-b94d-f5d50bb91a49" path="/var/lib/kubelet/pods/5c460f5c-a92c-4a10-b94d-f5d50bb91a49/volumes"
Sep 16 14:29:02.741493 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:02.741473    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:29:02.819110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:02.819088    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmfd6\" (UniqueName: \"kubernetes.io/projected/ca2a4982-7385-43c8-8b4a-7c4e6eca807b-kube-api-access-dmfd6\") pod \"ca2a4982-7385-43c8-8b4a-7c4e6eca807b\" (UID: \"ca2a4982-7385-43c8-8b4a-7c4e6eca807b\") "
Sep 16 14:29:02.820959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:02.820937    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca2a4982-7385-43c8-8b4a-7c4e6eca807b-kube-api-access-dmfd6" (OuterVolumeSpecName: "kube-api-access-dmfd6") pod "ca2a4982-7385-43c8-8b4a-7c4e6eca807b" (UID: "ca2a4982-7385-43c8-8b4a-7c4e6eca807b"). InnerVolumeSpecName "kube-api-access-dmfd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:29:02.920083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:02.920057    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmfd6\" (UniqueName: \"kubernetes.io/projected/ca2a4982-7385-43c8-8b4a-7c4e6eca807b-kube-api-access-dmfd6\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:29:02.971979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:02.971944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:03.486870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:03.486839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:03.594133 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:03.594103    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca2a4982-7385-43c8-8b4a-7c4e6eca807b" containerID="4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa" exitCode=137
Sep 16 14:29:03.594623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:03.594604    2580 scope.go:117] "RemoveContainer" containerID="4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa"
Sep 16 14:29:03.594815 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:03.594793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:29:03.604022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:03.604005    2580 scope.go:117] "RemoveContainer" containerID="4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa"
Sep 16 14:29:03.604305 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:29:03.604286    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa\": container with ID starting with 4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa not found: ID does not exist" containerID="4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa"
Sep 16 14:29:03.604355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:03.604312    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa"} err="failed to get container status \"4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa\": rpc error: code = NotFound desc = could not find container \"4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa\": container with ID starting with 4175b4ff682d4635281e508ab4bcd909c2b5ee98b51766fbc2cf10075ec4a1fa not found: ID does not exist"
Sep 16 14:29:03.604865 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:29:03.604848    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8956-2ba13e\" is forbidden: User \"system:node:ip-10-0-137-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-34.ec2.internal' and this object" podUID="ca2a4982-7385-43c8-8b4a-7c4e6eca807b" pod="opendatahub/test-curl-8956-2ba13e"
Sep 16 14:29:03.611645 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:03.611623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:04.013609 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:04.013575    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca2a4982-7385-43c8-8b4a-7c4e6eca807b" path="/var/lib/kubelet/pods/ca2a4982-7385-43c8-8b4a-7c4e6eca807b/volumes"
Sep 16 14:29:04.120866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:04.120833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:04.931926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:04.931878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:05.033303 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:05.033272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:05.938867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:05.938832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:06.872809 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:06.872776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:06.965323 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:06.965291    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:07.137991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.137919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:07.623272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.623230    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:07.716257 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.716225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:07.814118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.814091    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"]
Sep 16 14:29:07.814521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.814508    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2a4982-7385-43c8-8b4a-7c4e6eca807b" containerName="test-curl-8956-2ba13e"
Sep 16 14:29:07.814561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.814524    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2a4982-7385-43c8-8b4a-7c4e6eca807b" containerName="test-curl-8956-2ba13e"
Sep 16 14:29:07.814612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.814602    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2a4982-7385-43c8-8b4a-7c4e6eca807b" containerName="test-curl-8956-2ba13e"
Sep 16 14:29:07.819039 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.819023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:07.822860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.822837    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-dffae957-dockercfg-96gc9\""
Sep 16 14:29:07.822982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.822839    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-dffae957\""
Sep 16 14:29:07.828471 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.828449    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"]
Sep 16 14:29:07.882073 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.882016    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"]
Sep 16 14:29:07.885969 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.885953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:07.897921 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.897901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"]
Sep 16 14:29:07.965017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.964989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-plugins-config-volume\") pod \"payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") " pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:07.965111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.965029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpnfn\" (UniqueName: \"kubernetes.io/projected/a8a26815-31dd-427b-87cd-53c8186f4682-kube-api-access-cpnfn\") pod \"payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:07.965111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.965086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a26815-31dd-427b-87cd-53c8186f4682-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:07.965237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:07.965115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcc47\" (UniqueName: \"kubernetes.io/projected/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-kube-api-access-jcc47\") pod \"payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") " pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:08.066150 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.066115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-plugins-config-volume\") pod \"payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") " pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:08.066256 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.066173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpnfn\" (UniqueName: \"kubernetes.io/projected/a8a26815-31dd-427b-87cd-53c8186f4682-kube-api-access-cpnfn\") pod \"payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:08.066256 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.066237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a26815-31dd-427b-87cd-53c8186f4682-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:08.066325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.066261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcc47\" (UniqueName: \"kubernetes.io/projected/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-kube-api-access-jcc47\") pod \"payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") " pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:08.066979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.066956    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-plugins-config-volume\") pod \"payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") " pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:08.067017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.066974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a26815-31dd-427b-87cd-53c8186f4682-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:08.074699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.074676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpnfn\" (UniqueName: \"kubernetes.io/projected/a8a26815-31dd-427b-87cd-53c8186f4682-kube-api-access-cpnfn\") pod \"payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:08.074799 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.074783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcc47\" (UniqueName: \"kubernetes.io/projected/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-kube-api-access-jcc47\") pod \"payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") " pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:08.130368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.130341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:08.195826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.195799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:08.409403 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.407098    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"]
Sep 16 14:29:08.475582 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.475557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"]
Sep 16 14:29:08.476094 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:29:08.476068    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04dcee2c_7629_4d5d_9b25_b488c9b02cd4.slice/crio-491961f2809683644ff875b9836a344078716f5721c0dd8689f1ce251f6d2878 WatchSource:0}: Error finding container 491961f2809683644ff875b9836a344078716f5721c0dd8689f1ce251f6d2878: Status 404 returned error can't find the container with id 491961f2809683644ff875b9836a344078716f5721c0dd8689f1ce251f6d2878
Sep 16 14:29:08.591338 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.591301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:08.624295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.624267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd" event={"ID":"a8a26815-31dd-427b-87cd-53c8186f4682","Type":"ContainerStarted","Data":"a0e9bb259a3633d95ee9b9f323009c95e0f7626c58af710db33c6b6cc3d99021"}
Sep 16 14:29:08.624443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.624325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd" event={"ID":"a8a26815-31dd-427b-87cd-53c8186f4682","Type":"ContainerStarted","Data":"801d2fc8959eebbe453c0c33031db4deae8aac5acce27715daaccd8389698cb7"}
Sep 16 14:29:08.624443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.624357    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:08.625999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.625978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7" event={"ID":"04dcee2c-7629-4d5d-9b25-b488c9b02cd4","Type":"ContainerStarted","Data":"130350bca0c16adf1841b5ca3de4f43df0bad34eaf7343fcc3512d5cef896620"}
Sep 16 14:29:08.626113 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.626022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7" event={"ID":"04dcee2c-7629-4d5d-9b25-b488c9b02cd4","Type":"ContainerStarted","Data":"491961f2809683644ff875b9836a344078716f5721c0dd8689f1ce251f6d2878"}
Sep 16 14:29:08.626113 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.626070    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:08.644990 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.644944    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd" podStartSLOduration=1.6449295240000001 podStartE2EDuration="1.644929524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:29:08.644692589 +0000 UTC m=+1479.132702665" watchObservedRunningTime="2026-09-16 14:29:08.644929524 +0000 UTC m=+1479.132939601"
Sep 16 14:29:08.662968 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:08.662888    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7" podStartSLOduration=1.66287177 podStartE2EDuration="1.66287177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:29:08.662427745 +0000 UTC m=+1479.150437823" watchObservedRunningTime="2026-09-16 14:29:08.66287177 +0000 UTC m=+1479.150881861"
Sep 16 14:29:12.281761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:12.281721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:29:19.633854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:19.633824    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:29:19.634398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:19.634084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:29:21.388658 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:21.388619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:23.468498 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.468469    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"]
Sep 16 14:29:23.473215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.473197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.476931 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.476907    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"default-dockercfg-bz4h4\""
Sep 16 14:29:23.477035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.476945    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"openshift-service-ca.crt\""
Sep 16 14:29:23.477035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.476984    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"kube-root-ca.crt\""
Sep 16 14:29:23.477035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.476984    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"sub-test-model-f72402-kserve-self-signed-certs\""
Sep 16 14:29:23.480581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.480563    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"]
Sep 16 14:29:23.484910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.484885    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:23.520754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.520733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb7n2\" (UniqueName: \"kubernetes.io/projected/4c70d268-93ad-4f91-aa86-d7d94290d405-kube-api-access-kb7n2\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.520837 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.520764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c70d268-93ad-4f91-aa86-d7d94290d405-tls-certs\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.520837 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.520825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-home\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.520909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.520840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-tmp-dir\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.520909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.520860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-model-cache\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.520909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.520888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-dshm\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.621989 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.621968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb7n2\" (UniqueName: \"kubernetes.io/projected/4c70d268-93ad-4f91-aa86-d7d94290d405-kube-api-access-kb7n2\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.621997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c70d268-93ad-4f91-aa86-d7d94290d405-tls-certs\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.622044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-home\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.622060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-tmp-dir\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622087 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.622080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-model-cache\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.622109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-dshm\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622485 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.622465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-home\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622595 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.622480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-tmp-dir\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.622595 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.622506    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-model-cache\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.624282 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.624264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-dshm\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.624476 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.624460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c70d268-93ad-4f91-aa86-d7d94290d405-tls-certs\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.630441 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.630419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb7n2\" (UniqueName: \"kubernetes.io/projected/4c70d268-93ad-4f91-aa86-d7d94290d405-kube-api-access-kb7n2\") pod \"sub-test-model-f72402-kserve-858fffd445-d4cht\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") " pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:23.783559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:23.783509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:24.057937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:24.057912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"]
Sep 16 14:29:24.060074 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:29:24.060049    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c70d268_93ad_4f91_aa86_d7d94290d405.slice/crio-e275e3ab078e80d405d35911e78f5285cedf87a38cd3b0ff689fcb83fbe2882f WatchSource:0}: Error finding container e275e3ab078e80d405d35911e78f5285cedf87a38cd3b0ff689fcb83fbe2882f: Status 404 returned error can't find the container with id e275e3ab078e80d405d35911e78f5285cedf87a38cd3b0ff689fcb83fbe2882f
Sep 16 14:29:24.466289 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:24.466258    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:24.721849 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:24.721760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht" event={"ID":"4c70d268-93ad-4f91-aa86-d7d94290d405","Type":"ContainerStarted","Data":"5710df3adeaba41cb3759a22e105d822747ef49dd18414c242874b03ed64d995"}
Sep 16 14:29:24.721849 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:24.721810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht" event={"ID":"4c70d268-93ad-4f91-aa86-d7d94290d405","Type":"ContainerStarted","Data":"e275e3ab078e80d405d35911e78f5285cedf87a38cd3b0ff689fcb83fbe2882f"}
Sep 16 14:29:24.744554 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:24.744507    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht" podStartSLOduration=1.7444933790000001 podStartE2EDuration="1.744493379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:29:24.742608389 +0000 UTC m=+1495.230618475" watchObservedRunningTime="2026-09-16 14:29:24.744493379 +0000 UTC m=+1495.232503474"
Sep 16 14:29:28.072000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:28.071963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:29.432631 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:29.432593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:30.161241 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:30.161210    2580 scope.go:117] "RemoveContainer" containerID="cf530ab0c7996ee430676ea58c368c68f821a4b906efc2f5013d607f4dfc4ef4"
Sep 16 14:29:30.169260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:30.169240    2580 scope.go:117] "RemoveContainer" containerID="3680deaa8c18d6a700fff78360b91b912eaa67fc3a38526613de8aea6062d410"
Sep 16 14:29:30.176565 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:30.176548    2580 scope.go:117] "RemoveContainer" containerID="815237b5722684975ea26f969a9921f10f2dee5f57aae9d27939b42febc7c547"
Sep 16 14:29:30.183563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:30.183546    2580 scope.go:117] "RemoveContainer" containerID="34a315ddb45265bd9e3a0d9a3ad99a6a64986f194f379ecd03b72cf93f39d9cd"
Sep 16 14:29:30.672189 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:30.672125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:31.274288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:31.274255    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:29:31.294004 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:31.293961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:32.205948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:32.205907    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:32.946270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:32.946241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:33.543432 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:33.543399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:33.784071 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:33.784045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:33.784250 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:33.784087    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:33.796127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:33.796055    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:34.790298 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:34.790270    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:29:35.316234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:35.316185    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:35.416851 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:35.416814    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:36.672269 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:36.672240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:45.285730 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:45.285696    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:29:48.817579 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:48.817543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:29:57.000757 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:57.000679    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:29:58.000755 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:29:58.000720    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:30:00.152939 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.152903    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"]
Sep 16 14:30:00.157955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.157931    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"]
Sep 16 14:30:00.158099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.158045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"
Sep 16 14:30:00.161504 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.161486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"
Sep 16 14:30:00.161659 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.161642    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:30:00.169088 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.169063    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz"]
Sep 16 14:30:00.176436 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.176410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"
Sep 16 14:30:00.184229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.182374    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"]
Sep 16 14:30:00.184229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.183485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz"
Sep 16 14:30:00.187579 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.187553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"
Sep 16 14:30:00.188721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.188640    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"]
Sep 16 14:30:00.188721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.188678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"]
Sep 16 14:30:00.189302 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.189068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"
Sep 16 14:30:00.208478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.208003    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz"]
Sep 16 14:30:00.214884 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.214575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"]
Sep 16 14:30:00.230341 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.230304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz"
Sep 16 14:30:00.238362 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.238181    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"
Sep 16 14:30:00.470997 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.470974    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"]
Sep 16 14:30:00.473653 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:30:00.473623    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e646a4_7fd6_424d_9500_2dfbfecd6281.slice/crio-8e4dc76a11f9821dede9298b00de88918d9149dc9e25a8b3fc347f06878f5e4f WatchSource:0}: Error finding container 8e4dc76a11f9821dede9298b00de88918d9149dc9e25a8b3fc347f06878f5e4f: Status 404 returned error can't find the container with id 8e4dc76a11f9821dede9298b00de88918d9149dc9e25a8b3fc347f06878f5e4f
Sep 16 14:30:00.492885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.492865    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"]
Sep 16 14:30:00.493652 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:30:00.493625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0113f57_043a_4669_9b8d_86bba7831d8b.slice/crio-dd7cc5aef1e4c9a6d5f6491bb9ab07f35d8ca7e380ddfd6d2ed48e8b706a57ec WatchSource:0}: Error finding container dd7cc5aef1e4c9a6d5f6491bb9ab07f35d8ca7e380ddfd6d2ed48e8b706a57ec: Status 404 returned error can't find the container with id dd7cc5aef1e4c9a6d5f6491bb9ab07f35d8ca7e380ddfd6d2ed48e8b706a57ec
Sep 16 14:30:00.535197 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.535177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz"]
Sep 16 14:30:00.536300 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:30:00.536275    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353b09d1_4f25_498e_b7e9_62e7c8915ec3.slice/crio-dcb3a6116a4f2c0cd35fbf248bd4a47df7370cf808c4506641e81ac995c7b431 WatchSource:0}: Error finding container dcb3a6116a4f2c0cd35fbf248bd4a47df7370cf808c4506641e81ac995c7b431: Status 404 returned error can't find the container with id dcb3a6116a4f2c0cd35fbf248bd4a47df7370cf808c4506641e81ac995c7b431
Sep 16 14:30:00.554114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.554095    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"]
Sep 16 14:30:00.555588 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:30:00.555565    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6145853_ef62_4e27_b8ad_362437da0d94.slice/crio-bec76d14654136f8b8b509f9e161de6d36b319d16ea8308ff0abc8d466c2861c WatchSource:0}: Error finding container bec76d14654136f8b8b509f9e161de6d36b319d16ea8308ff0abc8d466c2861c: Status 404 returned error can't find the container with id bec76d14654136f8b8b509f9e161de6d36b319d16ea8308ff0abc8d466c2861c
Sep 16 14:30:00.921213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.921175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd" event={"ID":"d8e646a4-7fd6-424d-9500-2dfbfecd6281","Type":"ContainerStarted","Data":"8c819618a2c648dc5f9400877c0538d8ed942a1164ffc86dbffe229f3c5bd484"}
Sep 16 14:30:00.921319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.921240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd" event={"ID":"d8e646a4-7fd6-424d-9500-2dfbfecd6281","Type":"ContainerStarted","Data":"8e4dc76a11f9821dede9298b00de88918d9149dc9e25a8b3fc347f06878f5e4f"}
Sep 16 14:30:00.922688 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.922658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8" event={"ID":"d6145853-ef62-4e27-b8ad-362437da0d94","Type":"ContainerStarted","Data":"5076a2614873b32c3e657d15f070362a205a6a9aef79f0412f4186a92359f37f"}
Sep 16 14:30:00.922794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.922734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8" event={"ID":"d6145853-ef62-4e27-b8ad-362437da0d94","Type":"ContainerStarted","Data":"bec76d14654136f8b8b509f9e161de6d36b319d16ea8308ff0abc8d466c2861c"}
Sep 16 14:30:00.924045 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.924026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz" event={"ID":"353b09d1-4f25-498e-b7e9-62e7c8915ec3","Type":"ContainerStarted","Data":"eeb7bb210b31499789563d668a7af5365375c06bf587883c64e8691dc5106689"}
Sep 16 14:30:00.924120 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.924062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz" event={"ID":"353b09d1-4f25-498e-b7e9-62e7c8915ec3","Type":"ContainerStarted","Data":"dcb3a6116a4f2c0cd35fbf248bd4a47df7370cf808c4506641e81ac995c7b431"}
Sep 16 14:30:00.925566 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.925545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng" event={"ID":"a0113f57-043a-4669-9b8d-86bba7831d8b","Type":"ContainerStarted","Data":"5ca56c1502fb2c60b404e80d03bcc07708601754251e19831c2c4593ae2f9552"}
Sep 16 14:30:00.925653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.925578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng" event={"ID":"a0113f57-043a-4669-9b8d-86bba7831d8b","Type":"ContainerStarted","Data":"dd7cc5aef1e4c9a6d5f6491bb9ab07f35d8ca7e380ddfd6d2ed48e8b706a57ec"}
Sep 16 14:30:00.940420 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.940375    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd" podStartSLOduration=0.940358437 podStartE2EDuration="940.358437ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:00.939374634 +0000 UTC m=+1531.427384711" watchObservedRunningTime="2026-09-16 14:30:00.940358437 +0000 UTC m=+1531.428368513"
Sep 16 14:30:00.957946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.957912    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz" podStartSLOduration=0.957900442 podStartE2EDuration="957.900442ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:00.956299126 +0000 UTC m=+1531.444309202" watchObservedRunningTime="2026-09-16 14:30:00.957900442 +0000 UTC m=+1531.445910516"
Sep 16 14:30:00.971895 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.971857    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8" podStartSLOduration=0.971844491 podStartE2EDuration="971.844491ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:00.970168159 +0000 UTC m=+1531.458178237" watchObservedRunningTime="2026-09-16 14:30:00.971844491 +0000 UTC m=+1531.459854558"
Sep 16 14:30:00.987281 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:00.987248    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng" podStartSLOduration=0.987237549 podStartE2EDuration="987.237549ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:00.985133679 +0000 UTC m=+1531.473143754" watchObservedRunningTime="2026-09-16 14:30:00.987237549 +0000 UTC m=+1531.475247624"
Sep 16 14:30:01.470169 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.470122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:30:01.933390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.933304    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8e646a4-7fd6-424d-9500-2dfbfecd6281" containerID="8c819618a2c648dc5f9400877c0538d8ed942a1164ffc86dbffe229f3c5bd484" exitCode=0
Sep 16 14:30:01.933390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.933379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd" event={"ID":"d8e646a4-7fd6-424d-9500-2dfbfecd6281","Type":"ContainerDied","Data":"8c819618a2c648dc5f9400877c0538d8ed942a1164ffc86dbffe229f3c5bd484"}
Sep 16 14:30:01.934908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.934881    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6145853-ef62-4e27-b8ad-362437da0d94" containerID="5076a2614873b32c3e657d15f070362a205a6a9aef79f0412f4186a92359f37f" exitCode=0
Sep 16 14:30:01.935031 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.934950    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8" event={"ID":"d6145853-ef62-4e27-b8ad-362437da0d94","Type":"ContainerDied","Data":"5076a2614873b32c3e657d15f070362a205a6a9aef79f0412f4186a92359f37f"}
Sep 16 14:30:01.936411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.936389    2580 generic.go:356] "Generic (PLEG): container finished" podID="353b09d1-4f25-498e-b7e9-62e7c8915ec3" containerID="eeb7bb210b31499789563d668a7af5365375c06bf587883c64e8691dc5106689" exitCode=0
Sep 16 14:30:01.936478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.936459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz" event={"ID":"353b09d1-4f25-498e-b7e9-62e7c8915ec3","Type":"ContainerDied","Data":"eeb7bb210b31499789563d668a7af5365375c06bf587883c64e8691dc5106689"}
Sep 16 14:30:01.937731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.937712    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0113f57-043a-4669-9b8d-86bba7831d8b" containerID="5ca56c1502fb2c60b404e80d03bcc07708601754251e19831c2c4593ae2f9552" exitCode=0
Sep 16 14:30:01.937810 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:01.937790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng" event={"ID":"a0113f57-043a-4669-9b8d-86bba7831d8b","Type":"ContainerDied","Data":"5ca56c1502fb2c60b404e80d03bcc07708601754251e19831c2c4593ae2f9552"}
Sep 16 14:30:03.073090 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.073071    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz"
Sep 16 14:30:03.122888 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.122861    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"
Sep 16 14:30:03.157806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.157781    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"
Sep 16 14:30:03.162495 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.162477    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"
Sep 16 14:30:03.953285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.953239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd" event={"ID":"d8e646a4-7fd6-424d-9500-2dfbfecd6281","Type":"ContainerDied","Data":"8e4dc76a11f9821dede9298b00de88918d9149dc9e25a8b3fc347f06878f5e4f"}
Sep 16 14:30:03.953524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.953350    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e4dc76a11f9821dede9298b00de88918d9149dc9e25a8b3fc347f06878f5e4f"
Sep 16 14:30:03.953524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.953449    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"
Sep 16 14:30:03.955761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.955713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8" event={"ID":"d6145853-ef62-4e27-b8ad-362437da0d94","Type":"ContainerDied","Data":"bec76d14654136f8b8b509f9e161de6d36b319d16ea8308ff0abc8d466c2861c"}
Sep 16 14:30:03.955911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.955778    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bec76d14654136f8b8b509f9e161de6d36b319d16ea8308ff0abc8d466c2861c"
Sep 16 14:30:03.955911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.955867    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"
Sep 16 14:30:03.957873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.957840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz" event={"ID":"353b09d1-4f25-498e-b7e9-62e7c8915ec3","Type":"ContainerDied","Data":"dcb3a6116a4f2c0cd35fbf248bd4a47df7370cf808c4506641e81ac995c7b431"}
Sep 16 14:30:03.957992 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.957904    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcb3a6116a4f2c0cd35fbf248bd4a47df7370cf808c4506641e81ac995c7b431"
Sep 16 14:30:03.957992 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.957979    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-vhgkz"
Sep 16 14:30:03.960279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.960246    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"
Sep 16 14:30:03.960279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.960254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng" event={"ID":"a0113f57-043a-4669-9b8d-86bba7831d8b","Type":"ContainerDied","Data":"dd7cc5aef1e4c9a6d5f6491bb9ab07f35d8ca7e380ddfd6d2ed48e8b706a57ec"}
Sep 16 14:30:03.960431 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:03.960298    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd7cc5aef1e4c9a6d5f6491bb9ab07f35d8ca7e380ddfd6d2ed48e8b706a57ec"
Sep 16 14:30:07.989972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.989931    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"]
Sep 16 14:30:07.990456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990413    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353b09d1-4f25-498e-b7e9-62e7c8915ec3" containerName="cleanup"
Sep 16 14:30:07.990456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990431    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="353b09d1-4f25-498e-b7e9-62e7c8915ec3" containerName="cleanup"
Sep 16 14:30:07.990456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990447    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0113f57-043a-4669-9b8d-86bba7831d8b" containerName="cleanup"
Sep 16 14:30:07.990456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990454    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0113f57-043a-4669-9b8d-86bba7831d8b" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990463    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8e646a4-7fd6-424d-9500-2dfbfecd6281" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990468    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8e646a4-7fd6-424d-9500-2dfbfecd6281" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990478    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6145853-ef62-4e27-b8ad-362437da0d94" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990484    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6145853-ef62-4e27-b8ad-362437da0d94" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990557    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6145853-ef62-4e27-b8ad-362437da0d94" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990565    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="353b09d1-4f25-498e-b7e9-62e7c8915ec3" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990573    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0113f57-043a-4669-9b8d-86bba7831d8b" containerName="cleanup"
Sep 16 14:30:07.990679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.990580    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8e646a4-7fd6-424d-9500-2dfbfecd6281" containerName="cleanup"
Sep 16 14:30:07.993639 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.993613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"
Sep 16 14:30:07.996517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.996493    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:30:07.996643 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:07.996627    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:30:08.010246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.010226    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"]
Sep 16 14:30:08.024645 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.024612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ca8520-f081-457f-94af-f9e3484a8ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dffae957-qxcng\" (UID: \"f7ca8520-f081-457f-94af-f9e3484a8ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"
Sep 16 14:30:08.126075 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.126048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ca8520-f081-457f-94af-f9e3484a8ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dffae957-qxcng\" (UID: \"f7ca8520-f081-457f-94af-f9e3484a8ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"
Sep 16 14:30:08.126630 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.126610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ca8520-f081-457f-94af-f9e3484a8ccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dffae957-qxcng\" (UID: \"f7ca8520-f081-457f-94af-f9e3484a8ccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"
Sep 16 14:30:08.306991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.306932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"
Sep 16 14:30:08.325729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.325704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:08.579937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.579916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"]
Sep 16 14:30:08.581308 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:30:08.581278    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ca8520_f081_457f_94af_f9e3484a8ccc.slice/crio-d2331d6acae8f6170c7ab92917d78859877d13533a18f35546d918e2a8159b61 WatchSource:0}: Error finding container d2331d6acae8f6170c7ab92917d78859877d13533a18f35546d918e2a8159b61: Status 404 returned error can't find the container with id d2331d6acae8f6170c7ab92917d78859877d13533a18f35546d918e2a8159b61
Sep 16 14:30:08.996591 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.996566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng" event={"ID":"f7ca8520-f081-457f-94af-f9e3484a8ccc","Type":"ContainerStarted","Data":"045e107e3ed19ccc36c7a417caa97fb8e62d9a8d9c081f705c7751c9a029f215"}
Sep 16 14:30:08.996880 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:08.996611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng" event={"ID":"f7ca8520-f081-457f-94af-f9e3484a8ccc","Type":"ContainerStarted","Data":"d2331d6acae8f6170c7ab92917d78859877d13533a18f35546d918e2a8159b61"}
Sep 16 14:30:09.013737 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:09.013698    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng" podStartSLOduration=2.013683239 podStartE2EDuration="2.013683239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:09.011484709 +0000 UTC m=+1539.499494784" watchObservedRunningTime="2026-09-16 14:30:09.013683239 +0000 UTC m=+1539.501693315"
Sep 16 14:30:09.333296 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:09.333257    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:10.004345 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:10.004308    2580 generic.go:356] "Generic (PLEG): container finished" podID="f7ca8520-f081-457f-94af-f9e3484a8ccc" containerID="045e107e3ed19ccc36c7a417caa97fb8e62d9a8d9c081f705c7751c9a029f215" exitCode=0
Sep 16 14:30:10.006681 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:10.006652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng" event={"ID":"f7ca8520-f081-457f-94af-f9e3484a8ccc","Type":"ContainerDied","Data":"045e107e3ed19ccc36c7a417caa97fb8e62d9a8d9c081f705c7751c9a029f215"}
Sep 16 14:30:10.060379 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:10.060353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:10.896688 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:10.896630    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:11.138727 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:11.138705    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"
Sep 16 14:30:11.256067 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:11.256043    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ca8520-f081-457f-94af-f9e3484a8ccc-maas-api-service-ca\") pod \"f7ca8520-f081-457f-94af-f9e3484a8ccc\" (UID: \"f7ca8520-f081-457f-94af-f9e3484a8ccc\") "
Sep 16 14:30:11.256355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:11.256333    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ca8520-f081-457f-94af-f9e3484a8ccc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7ca8520-f081-457f-94af-f9e3484a8ccc" (UID: "f7ca8520-f081-457f-94af-f9e3484a8ccc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:30:11.357064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:11.357042    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ca8520-f081-457f-94af-f9e3484a8ccc-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:11.834257 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:11.834219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:12.017200 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:12.017173    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"
Sep 16 14:30:12.017200 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:12.017178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng" event={"ID":"f7ca8520-f081-457f-94af-f9e3484a8ccc","Type":"ContainerDied","Data":"d2331d6acae8f6170c7ab92917d78859877d13533a18f35546d918e2a8159b61"}
Sep 16 14:30:12.017359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:12.017228    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2331d6acae8f6170c7ab92917d78859877d13533a18f35546d918e2a8159b61"
Sep 16 14:30:12.583679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:12.583649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:18.888114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:18.888076    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:30:23.833372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:23.833344    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:23.877956 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:23.877927    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"]
Sep 16 14:30:23.878232 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:23.878205    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7" podUID="04dcee2c-7629-4d5d-9b25-b488c9b02cd4" containerName="payload-processing" containerID="cri-o://130350bca0c16adf1841b5ca3de4f43df0bad34eaf7343fcc3512d5cef896620" gracePeriod=30
Sep 16 14:30:23.895221 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:23.894648    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"]
Sep 16 14:30:23.895221 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:23.894949    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd" podUID="a8a26815-31dd-427b-87cd-53c8186f4682" containerName="payload-pre-processing" containerID="cri-o://a0e9bb259a3633d95ee9b9f323009c95e0f7626c58af710db33c6b6cc3d99021" gracePeriod=30
Sep 16 14:30:24.091980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.091721    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8a26815-31dd-427b-87cd-53c8186f4682" containerID="a0e9bb259a3633d95ee9b9f323009c95e0f7626c58af710db33c6b6cc3d99021" exitCode=0
Sep 16 14:30:24.091980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.091863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd" event={"ID":"a8a26815-31dd-427b-87cd-53c8186f4682","Type":"ContainerDied","Data":"a0e9bb259a3633d95ee9b9f323009c95e0f7626c58af710db33c6b6cc3d99021"}
Sep 16 14:30:24.095192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.094962    2580 generic.go:356] "Generic (PLEG): container finished" podID="04dcee2c-7629-4d5d-9b25-b488c9b02cd4" containerID="130350bca0c16adf1841b5ca3de4f43df0bad34eaf7343fcc3512d5cef896620" exitCode=0
Sep 16 14:30:24.095192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.095097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7" event={"ID":"04dcee2c-7629-4d5d-9b25-b488c9b02cd4","Type":"ContainerDied","Data":"130350bca0c16adf1841b5ca3de4f43df0bad34eaf7343fcc3512d5cef896620"}
Sep 16 14:30:24.107426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.107401    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"]
Sep 16 14:30:24.109682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.109659    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dffae957-qxcng"]
Sep 16 14:30:24.171073 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.171052    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:30:24.194664 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.194644    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:30:24.275372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.275343    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a26815-31dd-427b-87cd-53c8186f4682-plugins-config-volume\") pod \"a8a26815-31dd-427b-87cd-53c8186f4682\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") "
Sep 16 14:30:24.275498 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.275397    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcc47\" (UniqueName: \"kubernetes.io/projected/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-kube-api-access-jcc47\") pod \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") "
Sep 16 14:30:24.275498 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.275476    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpnfn\" (UniqueName: \"kubernetes.io/projected/a8a26815-31dd-427b-87cd-53c8186f4682-kube-api-access-cpnfn\") pod \"a8a26815-31dd-427b-87cd-53c8186f4682\" (UID: \"a8a26815-31dd-427b-87cd-53c8186f4682\") "
Sep 16 14:30:24.275617 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.275527    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-plugins-config-volume\") pod \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\" (UID: \"04dcee2c-7629-4d5d-9b25-b488c9b02cd4\") "
Sep 16 14:30:24.275775 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.275747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8a26815-31dd-427b-87cd-53c8186f4682-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8a26815-31dd-427b-87cd-53c8186f4682" (UID: "a8a26815-31dd-427b-87cd-53c8186f4682"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:30:24.275943 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.275907    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a26815-31dd-427b-87cd-53c8186f4682-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:24.276019 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.275982    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04dcee2c-7629-4d5d-9b25-b488c9b02cd4" (UID: "04dcee2c-7629-4d5d-9b25-b488c9b02cd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:30:24.277578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.277557    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-kube-api-access-jcc47" (OuterVolumeSpecName: "kube-api-access-jcc47") pod "04dcee2c-7629-4d5d-9b25-b488c9b02cd4" (UID: "04dcee2c-7629-4d5d-9b25-b488c9b02cd4"). InnerVolumeSpecName "kube-api-access-jcc47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:30:24.277665 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.277579    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8a26815-31dd-427b-87cd-53c8186f4682-kube-api-access-cpnfn" (OuterVolumeSpecName: "kube-api-access-cpnfn") pod "a8a26815-31dd-427b-87cd-53c8186f4682" (UID: "a8a26815-31dd-427b-87cd-53c8186f4682"). InnerVolumeSpecName "kube-api-access-cpnfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:30:24.376555 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.376505    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcc47\" (UniqueName: \"kubernetes.io/projected/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-kube-api-access-jcc47\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:24.376555 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.376525    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpnfn\" (UniqueName: \"kubernetes.io/projected/a8a26815-31dd-427b-87cd-53c8186f4682-kube-api-access-cpnfn\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:24.376555 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:24.376534    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04dcee2c-7629-4d5d-9b25-b488c9b02cd4-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:25.101828 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.101801    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"
Sep 16 14:30:25.102214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.101845    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7" event={"ID":"04dcee2c-7629-4d5d-9b25-b488c9b02cd4","Type":"ContainerDied","Data":"491961f2809683644ff875b9836a344078716f5721c0dd8689f1ce251f6d2878"}
Sep 16 14:30:25.102214 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.101902    2580 scope.go:117] "RemoveContainer" containerID="130350bca0c16adf1841b5ca3de4f43df0bad34eaf7343fcc3512d5cef896620"
Sep 16 14:30:25.103478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.103460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd" event={"ID":"a8a26815-31dd-427b-87cd-53c8186f4682","Type":"ContainerDied","Data":"801d2fc8959eebbe453c0c33031db4deae8aac5acce27715daaccd8389698cb7"}
Sep 16 14:30:25.103571 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.103488    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"
Sep 16 14:30:25.111307 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.111290    2580 scope.go:117] "RemoveContainer" containerID="a0e9bb259a3633d95ee9b9f323009c95e0f7626c58af710db33c6b6cc3d99021"
Sep 16 14:30:25.128371 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.128351    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"]
Sep 16 14:30:25.132390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.132370    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dffae957-69f54f5c49-hc7f7"]
Sep 16 14:30:25.145412 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.145389    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"]
Sep 16 14:30:25.147885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.147868    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dffae957-868ddcd954-fcnxd"]
Sep 16 14:30:25.732410 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:25.732376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:26.007545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:26.007471    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04dcee2c-7629-4d5d-9b25-b488c9b02cd4" path="/var/lib/kubelet/pods/04dcee2c-7629-4d5d-9b25-b488c9b02cd4/volumes"
Sep 16 14:30:26.007816 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:26.007804    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8a26815-31dd-427b-87cd-53c8186f4682" path="/var/lib/kubelet/pods/a8a26815-31dd-427b-87cd-53c8186f4682/volumes"
Sep 16 14:30:26.008080 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:26.008069    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ca8520-f081-457f-94af-f9e3484a8ccc" path="/var/lib/kubelet/pods/f7ca8520-f081-457f-94af-f9e3484a8ccc/volumes"
Sep 16 14:30:26.554278 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:26.554240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:27.277296 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:27.277261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:27.378147 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:27.378105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:30.237708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:30.237678    2580 scope.go:117] "RemoveContainer" containerID="0fa7c76f1a50cb45f689c133926b8c5f75edb9a26a433ae4cdedfec300c802ac"
Sep 16 14:30:32.335940 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:32.335894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:32.477789 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:32.477759    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:30:38.222811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.222779    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8959-093161"]
Sep 16 14:30:38.223225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223210    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7ca8520-f081-457f-94af-f9e3484a8ccc" containerName="revoke-keys"
Sep 16 14:30:38.223277 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223236    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ca8520-f081-457f-94af-f9e3484a8ccc" containerName="revoke-keys"
Sep 16 14:30:38.223277 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223251    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8a26815-31dd-427b-87cd-53c8186f4682" containerName="payload-pre-processing"
Sep 16 14:30:38.223277 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223257    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8a26815-31dd-427b-87cd-53c8186f4682" containerName="payload-pre-processing"
Sep 16 14:30:38.223381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223278    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04dcee2c-7629-4d5d-9b25-b488c9b02cd4" containerName="payload-processing"
Sep 16 14:30:38.223381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223284    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04dcee2c-7629-4d5d-9b25-b488c9b02cd4" containerName="payload-processing"
Sep 16 14:30:38.223381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223353    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7ca8520-f081-457f-94af-f9e3484a8ccc" containerName="revoke-keys"
Sep 16 14:30:38.223381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223366    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04dcee2c-7629-4d5d-9b25-b488c9b02cd4" containerName="payload-processing"
Sep 16 14:30:38.223381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.223373    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8a26815-31dd-427b-87cd-53c8186f4682" containerName="payload-pre-processing"
Sep 16 14:30:38.226862 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.226847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-093161"
Sep 16 14:30:38.229512 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.229489    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8t2sn\""
Sep 16 14:30:38.232644 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.232621    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8959-093161"]
Sep 16 14:30:38.301677 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.301652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mld8r\" (UniqueName: \"kubernetes.io/projected/4d933ce9-9ddb-41f0-be76-dc267a94f597-kube-api-access-mld8r\") pod \"test-curl-8959-093161\" (UID: \"4d933ce9-9ddb-41f0-be76-dc267a94f597\") " pod="opendatahub/test-curl-8959-093161"
Sep 16 14:30:38.402476 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.402450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mld8r\" (UniqueName: \"kubernetes.io/projected/4d933ce9-9ddb-41f0-be76-dc267a94f597-kube-api-access-mld8r\") pod \"test-curl-8959-093161\" (UID: \"4d933ce9-9ddb-41f0-be76-dc267a94f597\") " pod="opendatahub/test-curl-8959-093161"
Sep 16 14:30:38.412843 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.412821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mld8r\" (UniqueName: \"kubernetes.io/projected/4d933ce9-9ddb-41f0-be76-dc267a94f597-kube-api-access-mld8r\") pod \"test-curl-8959-093161\" (UID: \"4d933ce9-9ddb-41f0-be76-dc267a94f597\") " pod="opendatahub/test-curl-8959-093161"
Sep 16 14:30:38.536628 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.536562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-093161"
Sep 16 14:30:38.805500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:38.805474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8959-093161"]
Sep 16 14:30:38.807521 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:30:38.807498    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d933ce9_9ddb_41f0_be76_dc267a94f597.slice/crio-dff5460d626d94b4b8f0318693be3bfefb13b8a6fabf8d04a9e5d96b92ea1be8 WatchSource:0}: Error finding container dff5460d626d94b4b8f0318693be3bfefb13b8a6fabf8d04a9e5d96b92ea1be8: Status 404 returned error can't find the container with id dff5460d626d94b4b8f0318693be3bfefb13b8a6fabf8d04a9e5d96b92ea1be8
Sep 16 14:30:39.182240 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:39.182167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8959-093161" event={"ID":"4d933ce9-9ddb-41f0-be76-dc267a94f597","Type":"ContainerStarted","Data":"00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188"}
Sep 16 14:30:39.182240 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:39.182210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8959-093161" event={"ID":"4d933ce9-9ddb-41f0-be76-dc267a94f597","Type":"ContainerStarted","Data":"dff5460d626d94b4b8f0318693be3bfefb13b8a6fabf8d04a9e5d96b92ea1be8"}
Sep 16 14:30:39.199991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:39.199950    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8959-093161" podStartSLOduration=1.199938065 podStartE2EDuration="1.199938065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:39.19791725 +0000 UTC m=+1569.685927331" watchObservedRunningTime="2026-09-16 14:30:39.199938065 +0000 UTC m=+1569.687948139"
Sep 16 14:30:39.815159 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:39.815107    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8959-093161"]
Sep 16 14:30:39.817861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:39.817836    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8959-093161"]
Sep 16 14:30:40.187217 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:30:40.187191    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8959-093161" container="test-curl-8959-093161"
Sep 16 14:30:41.192114 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:41.192077    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8959-093161" podUID="4d933ce9-9ddb-41f0-be76-dc267a94f597" containerName="test-curl-8959-093161" containerID="cri-o://00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188" gracePeriod=2
Sep 16 14:30:41.872724 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:41.872689    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"]
Sep 16 14:30:41.896094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:41.896067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:43.064638 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:43.064610    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"]
Sep 16 14:30:43.082786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:43.082761    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:43.393661 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:43.393639    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-093161"
Sep 16 14:30:43.453426 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:43.453400    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mld8r\" (UniqueName: \"kubernetes.io/projected/4d933ce9-9ddb-41f0-be76-dc267a94f597-kube-api-access-mld8r\") pod \"4d933ce9-9ddb-41f0-be76-dc267a94f597\" (UID: \"4d933ce9-9ddb-41f0-be76-dc267a94f597\") "
Sep 16 14:30:43.455258 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:43.455239    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d933ce9-9ddb-41f0-be76-dc267a94f597-kube-api-access-mld8r" (OuterVolumeSpecName: "kube-api-access-mld8r") pod "4d933ce9-9ddb-41f0-be76-dc267a94f597" (UID: "4d933ce9-9ddb-41f0-be76-dc267a94f597"). InnerVolumeSpecName "kube-api-access-mld8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:30:43.554940 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:43.554916    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mld8r\" (UniqueName: \"kubernetes.io/projected/4d933ce9-9ddb-41f0-be76-dc267a94f597-kube-api-access-mld8r\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:44.006308 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:44.006271    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d933ce9-9ddb-41f0-be76-dc267a94f597" path="/var/lib/kubelet/pods/4d933ce9-9ddb-41f0-be76-dc267a94f597/volumes"
Sep 16 14:30:44.208092 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:44.208068    2580 generic.go:356] "Generic (PLEG): container finished" podID="4d933ce9-9ddb-41f0-be76-dc267a94f597" containerID="00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188" exitCode=137
Sep 16 14:30:44.208452 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:44.208107    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-093161"
Sep 16 14:30:44.208452 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:44.208181    2580 scope.go:117] "RemoveContainer" containerID="00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188"
Sep 16 14:30:44.216647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:44.216628    2580 scope.go:117] "RemoveContainer" containerID="00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188"
Sep 16 14:30:44.216913 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:30:44.216895    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188\": container with ID starting with 00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188 not found: ID does not exist" containerID="00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188"
Sep 16 14:30:44.216979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:44.216925    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188"} err="failed to get container status \"00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188\": rpc error: code = NotFound desc = could not find container \"00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188\": container with ID starting with 00e49e4d77b2fa5b715e0e85b35b66e8aca230e9e24ed8351de8ee78199aa188 not found: ID does not exist"
Sep 16 14:30:45.983567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:45.983529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:30:51.669891 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.669854    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"]
Sep 16 14:30:51.670399 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.670385    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d933ce9-9ddb-41f0-be76-dc267a94f597" containerName="test-curl-8959-093161"
Sep 16 14:30:51.670479 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.670404    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d933ce9-9ddb-41f0-be76-dc267a94f597" containerName="test-curl-8959-093161"
Sep 16 14:30:51.670546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.670491    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d933ce9-9ddb-41f0-be76-dc267a94f597" containerName="test-curl-8959-093161"
Sep 16 14:30:51.675007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.674976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.677420 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.677396    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-50b57c\"/\"rate-test-model-50b57c-kserve-self-signed-certs\""
Sep 16 14:30:51.683768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.683746    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"]
Sep 16 14:30:51.703427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.703400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:51.724835 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.724816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-model-cache\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.724933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.724878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdrld\" (UniqueName: \"kubernetes.io/projected/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-kube-api-access-bdrld\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.724933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.724911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tls-certs\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.724933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.724929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-home\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.725066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.724989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tmp-dir\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.725066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.725046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-dshm\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826348 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-model-cache\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdrld\" (UniqueName: \"kubernetes.io/projected/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-kube-api-access-bdrld\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tls-certs\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-home\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tmp-dir\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-dshm\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-model-cache\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826872 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-home\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.826872 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.826860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tmp-dir\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.828713 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.828689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-dshm\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.828985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.828966    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tls-certs\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.834966 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.834940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdrld\" (UniqueName: \"kubernetes.io/projected/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-kube-api-access-bdrld\") pod \"rate-test-model-50b57c-kserve-7c4958b987-tpccp\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") " pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:51.986015 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:51.985991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:30:52.261756 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:52.261730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"]
Sep 16 14:30:52.263308 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:30:52.263280    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad17a8a4_4d80_4fa2_b5d1_e660816f9fa4.slice/crio-07f7b754faf8633a43bce54367ea130f1d4ab64718e9db4364202127aca74404 WatchSource:0}: Error finding container 07f7b754faf8633a43bce54367ea130f1d4ab64718e9db4364202127aca74404: Status 404 returned error can't find the container with id 07f7b754faf8633a43bce54367ea130f1d4ab64718e9db4364202127aca74404
Sep 16 14:30:53.258180 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:53.258127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp" event={"ID":"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4","Type":"ContainerStarted","Data":"80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e"}
Sep 16 14:30:53.258180 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:53.258187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp" event={"ID":"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4","Type":"ContainerStarted","Data":"07f7b754faf8633a43bce54367ea130f1d4ab64718e9db4364202127aca74404"}
Sep 16 14:30:53.278770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:53.278714    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp" podStartSLOduration=2.278700777 podStartE2EDuration="2.278700777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:53.276558282 +0000 UTC m=+1583.764568380" watchObservedRunningTime="2026-09-16 14:30:53.278700777 +0000 UTC m=+1583.766710853"
Sep 16 14:30:55.404350 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:55.404313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:30:56.890775 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:56.890735    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5" podUID="ee31a4e0-85f2-48d2-91e3-821cadb7984c" containerName="main" containerID="cri-o://7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63" gracePeriod=45
Sep 16 14:30:57.138937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.138914    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:30:57.274608 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.274583    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-dshm\") pod \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") "
Sep 16 14:30:57.274780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.274649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tmp-dir\") pod \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") "
Sep 16 14:30:57.274780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.274716    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-model-cache\") pod \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") "
Sep 16 14:30:57.274780 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.274739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-home\") pod \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") "
Sep 16 14:30:57.274945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.274791    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tls-certs\") pod \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") "
Sep 16 14:30:57.274945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.274841    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmrrn\" (UniqueName: \"kubernetes.io/projected/ee31a4e0-85f2-48d2-91e3-821cadb7984c-kube-api-access-vmrrn\") pod \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\" (UID: \"ee31a4e0-85f2-48d2-91e3-821cadb7984c\") "
Sep 16 14:30:57.275041 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.274978    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee31a4e0-85f2-48d2-91e3-821cadb7984c" (UID: "ee31a4e0-85f2-48d2-91e3-821cadb7984c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:57.275041 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.275024    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-home" (OuterVolumeSpecName: "home") pod "ee31a4e0-85f2-48d2-91e3-821cadb7984c" (UID: "ee31a4e0-85f2-48d2-91e3-821cadb7984c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:57.275126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.275055    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee31a4e0-85f2-48d2-91e3-821cadb7984c" (UID: "ee31a4e0-85f2-48d2-91e3-821cadb7984c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:57.275278 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.275254    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:57.275278 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.275280    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:57.275469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.275294    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:57.276978 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.276952    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee31a4e0-85f2-48d2-91e3-821cadb7984c" (UID: "ee31a4e0-85f2-48d2-91e3-821cadb7984c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:30:57.277094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.277073    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-dshm" (OuterVolumeSpecName: "dshm") pod "ee31a4e0-85f2-48d2-91e3-821cadb7984c" (UID: "ee31a4e0-85f2-48d2-91e3-821cadb7984c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:57.277186 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.277132    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee31a4e0-85f2-48d2-91e3-821cadb7984c-kube-api-access-vmrrn" (OuterVolumeSpecName: "kube-api-access-vmrrn") pod "ee31a4e0-85f2-48d2-91e3-821cadb7984c" (UID: "ee31a4e0-85f2-48d2-91e3-821cadb7984c"). InnerVolumeSpecName "kube-api-access-vmrrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:30:57.279662 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.279636    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee31a4e0-85f2-48d2-91e3-821cadb7984c" containerID="7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63" exitCode=0
Sep 16 14:30:57.279778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.279668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5" event={"ID":"ee31a4e0-85f2-48d2-91e3-821cadb7984c","Type":"ContainerDied","Data":"7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63"}
Sep 16 14:30:57.279778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.279678    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"
Sep 16 14:30:57.279778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.279710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5" event={"ID":"ee31a4e0-85f2-48d2-91e3-821cadb7984c","Type":"ContainerDied","Data":"e63c693fcfe3a2fb5ec2e3a04b341871f87b4416741fa08a3978770d047cc822"}
Sep 16 14:30:57.279778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.279726    2580 scope.go:117] "RemoveContainer" containerID="7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63"
Sep 16 14:30:57.297033 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.297018    2580 scope.go:117] "RemoveContainer" containerID="7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63"
Sep 16 14:30:57.297322 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:30:57.297305    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63\": container with ID starting with 7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63 not found: ID does not exist" containerID="7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63"
Sep 16 14:30:57.297386 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.297331    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63"} err="failed to get container status \"7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63\": rpc error: code = NotFound desc = could not find container \"7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63\": container with ID starting with 7b75f0b079fe03b89921aa3c3005c8cc3173f28f70ae44d8be81346f0b2cea63 not found: ID does not exist"
Sep 16 14:30:57.310548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.310526    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"]
Sep 16 14:30:57.314282 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.314259    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/sub-test-model-50b57c-kserve-77bb58c787-8tmd5"]
Sep 16 14:30:57.375793 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.375775    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee31a4e0-85f2-48d2-91e3-821cadb7984c-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:57.375873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.375795    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmrrn\" (UniqueName: \"kubernetes.io/projected/ee31a4e0-85f2-48d2-91e3-821cadb7984c-kube-api-access-vmrrn\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:57.375873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:57.375805    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee31a4e0-85f2-48d2-91e3-821cadb7984c-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:58.006354 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.006324    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee31a4e0-85f2-48d2-91e3-821cadb7984c" path="/var/lib/kubelet/pods/ee31a4e0-85f2-48d2-91e3-821cadb7984c/volumes"
Sep 16 14:30:58.082897 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.082858    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht" podUID="4c70d268-93ad-4f91-aa86-d7d94290d405" containerName="main" containerID="cri-o://5710df3adeaba41cb3759a22e105d822747ef49dd18414c242874b03ed64d995" gracePeriod=45
Sep 16 14:30:58.287457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.287389    2580 generic.go:356] "Generic (PLEG): container finished" podID="4c70d268-93ad-4f91-aa86-d7d94290d405" containerID="5710df3adeaba41cb3759a22e105d822747ef49dd18414c242874b03ed64d995" exitCode=0
Sep 16 14:30:58.287613 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.287461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht" event={"ID":"4c70d268-93ad-4f91-aa86-d7d94290d405","Type":"ContainerDied","Data":"5710df3adeaba41cb3759a22e105d822747ef49dd18414c242874b03ed64d995"}
Sep 16 14:30:58.336249 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.336230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:30:58.485517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485485    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb7n2\" (UniqueName: \"kubernetes.io/projected/4c70d268-93ad-4f91-aa86-d7d94290d405-kube-api-access-kb7n2\") pod \"4c70d268-93ad-4f91-aa86-d7d94290d405\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") "
Sep 16 14:30:58.485687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c70d268-93ad-4f91-aa86-d7d94290d405-tls-certs\") pod \"4c70d268-93ad-4f91-aa86-d7d94290d405\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") "
Sep 16 14:30:58.485687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485585    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-tmp-dir\") pod \"4c70d268-93ad-4f91-aa86-d7d94290d405\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") "
Sep 16 14:30:58.485687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485638    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-dshm\") pod \"4c70d268-93ad-4f91-aa86-d7d94290d405\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") "
Sep 16 14:30:58.485687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485669    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-home\") pod \"4c70d268-93ad-4f91-aa86-d7d94290d405\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") "
Sep 16 14:30:58.485687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485685    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-model-cache\") pod \"4c70d268-93ad-4f91-aa86-d7d94290d405\" (UID: \"4c70d268-93ad-4f91-aa86-d7d94290d405\") "
Sep 16 14:30:58.485957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c70d268-93ad-4f91-aa86-d7d94290d405" (UID: "4c70d268-93ad-4f91-aa86-d7d94290d405"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:58.485957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.485941    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-home" (OuterVolumeSpecName: "home") pod "4c70d268-93ad-4f91-aa86-d7d94290d405" (UID: "4c70d268-93ad-4f91-aa86-d7d94290d405"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:58.486049 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.486028    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c70d268-93ad-4f91-aa86-d7d94290d405" (UID: "4c70d268-93ad-4f91-aa86-d7d94290d405"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:58.487690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.487668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-dshm" (OuterVolumeSpecName: "dshm") pod "4c70d268-93ad-4f91-aa86-d7d94290d405" (UID: "4c70d268-93ad-4f91-aa86-d7d94290d405"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:58.487817 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.487692    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c70d268-93ad-4f91-aa86-d7d94290d405-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c70d268-93ad-4f91-aa86-d7d94290d405" (UID: "4c70d268-93ad-4f91-aa86-d7d94290d405"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:30:58.487817 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.487756    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c70d268-93ad-4f91-aa86-d7d94290d405-kube-api-access-kb7n2" (OuterVolumeSpecName: "kube-api-access-kb7n2") pod "4c70d268-93ad-4f91-aa86-d7d94290d405" (UID: "4c70d268-93ad-4f91-aa86-d7d94290d405"). InnerVolumeSpecName "kube-api-access-kb7n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:30:58.586892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.586838    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:58.586892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.586859    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:58.586892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.586867    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:58.586892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.586878    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb7n2\" (UniqueName: \"kubernetes.io/projected/4c70d268-93ad-4f91-aa86-d7d94290d405-kube-api-access-kb7n2\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:58.586892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.586885    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c70d268-93ad-4f91-aa86-d7d94290d405-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:58.586892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:58.586894    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c70d268-93ad-4f91-aa86-d7d94290d405-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:30:59.294987 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:59.294956    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"
Sep 16 14:30:59.295440 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:59.294988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht" event={"ID":"4c70d268-93ad-4f91-aa86-d7d94290d405","Type":"ContainerDied","Data":"e275e3ab078e80d405d35911e78f5285cedf87a38cd3b0ff689fcb83fbe2882f"}
Sep 16 14:30:59.295440 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:59.295047    2580 scope.go:117] "RemoveContainer" containerID="5710df3adeaba41cb3759a22e105d822747ef49dd18414c242874b03ed64d995"
Sep 16 14:30:59.317086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:59.317062    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"]
Sep 16 14:30:59.320802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:30:59.320782    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/sub-test-model-f72402-kserve-858fffd445-d4cht"]
Sep 16 14:31:00.012793 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:00.012751    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c70d268-93ad-4f91-aa86-d7d94290d405" path="/var/lib/kubelet/pods/4c70d268-93ad-4f91-aa86-d7d94290d405/volumes"
Sep 16 14:31:01.986127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:01.986093    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:31:01.986661 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:01.986155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:31:01.998502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:01.998473    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:31:02.322302 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:02.322216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:31:04.091392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:04.091357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:31:06.000766 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:06.000732    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:31:08.908477 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:08.908440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:10.643183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:10.643151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:17.517302 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:17.517260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:17.659238 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:17.659203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:23.715668 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:23.715584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:23.843603 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:23.843571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:24.044625 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:24.044532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:25.509993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:25.509963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:25.611323 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:25.611288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:26.700688 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:26.700653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:26.957845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:26.957758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:27.000046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:27.000015    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:31:28.361246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:28.361208    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:29.916614 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:29.916577    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:31:30.266194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:30.266165    2580 scope.go:117] "RemoveContainer" containerID="1579715c8d43a32f0a4363c9cd6739f78fe8e4717bd4772e357142d271028acb"
Sep 16 14:31:30.275007 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:30.274986    2580 scope.go:117] "RemoveContainer" containerID="60d0814c050522f499c483b2eb264346cc9db596290574d78f718003aef85e4c"
Sep 16 14:31:35.246764 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:35.246726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:35.364071 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:35.364040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:40.776456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.776419    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"]
Sep 16 14:31:40.776858 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.776846    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee31a4e0-85f2-48d2-91e3-821cadb7984c" containerName="main"
Sep 16 14:31:40.776906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.776860    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee31a4e0-85f2-48d2-91e3-821cadb7984c" containerName="main"
Sep 16 14:31:40.776906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.776870    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c70d268-93ad-4f91-aa86-d7d94290d405" containerName="main"
Sep 16 14:31:40.776906 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.776875    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c70d268-93ad-4f91-aa86-d7d94290d405" containerName="main"
Sep 16 14:31:40.777014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.776935    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c70d268-93ad-4f91-aa86-d7d94290d405" containerName="main"
Sep 16 14:31:40.777014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.776944    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee31a4e0-85f2-48d2-91e3-821cadb7984c" containerName="main"
Sep 16 14:31:40.779968 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.779951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.784856 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.784835    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"openshift-service-ca.crt\""
Sep 16 14:31:40.785072 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.785055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"rate-test-model-f72402-kserve-self-signed-certs\""
Sep 16 14:31:40.785252 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.785237    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"kube-root-ca.crt\""
Sep 16 14:31:40.785332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.785269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f72402\"/\"default-dockercfg-bz4h4\""
Sep 16 14:31:40.792621 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.792599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"]
Sep 16 14:31:40.808378 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.808349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:31:40.890105 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.890072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-model-cache\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.890292 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.890116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-home\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.890292 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.890236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-dshm\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.890413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.890289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2j9m\" (UniqueName: \"kubernetes.io/projected/b3057ab4-9bf2-493e-8e63-3acb575cbc46-kube-api-access-k2j9m\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.890413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.890334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tls-certs\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.890413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.890402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tmp-dir\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-home\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-dshm\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2j9m\" (UniqueName: \"kubernetes.io/projected/b3057ab4-9bf2-493e-8e63-3acb575cbc46-kube-api-access-k2j9m\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tls-certs\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tmp-dir\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-model-cache\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991939 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-home\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.991985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.991944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-model-cache\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.992096 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.992066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tmp-dir\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.993732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.993702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-dshm\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:40.994020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:40.994003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tls-certs\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:41.000310 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:41.000288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2j9m\" (UniqueName: \"kubernetes.io/projected/b3057ab4-9bf2-493e-8e63-3acb575cbc46-kube-api-access-k2j9m\") pod \"rate-test-model-f72402-kserve-ff7f6f559-vjwtd\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") " pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:41.089434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:41.089365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:41.365546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:41.365516    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"]
Sep 16 14:31:41.367019 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:31:41.366991    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3057ab4_9bf2_493e_8e63_3acb575cbc46.slice/crio-ca2fb850efe761fb943bace4954dd9e17d24ae3c8c361ef27a9f6c6446bf8fcf WatchSource:0}: Error finding container ca2fb850efe761fb943bace4954dd9e17d24ae3c8c361ef27a9f6c6446bf8fcf: Status 404 returned error can't find the container with id ca2fb850efe761fb943bace4954dd9e17d24ae3c8c361ef27a9f6c6446bf8fcf
Sep 16 14:31:41.526466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:41.526433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd" event={"ID":"b3057ab4-9bf2-493e-8e63-3acb575cbc46","Type":"ContainerStarted","Data":"b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e"}
Sep 16 14:31:41.526466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:41.526471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd" event={"ID":"b3057ab4-9bf2-493e-8e63-3acb575cbc46","Type":"ContainerStarted","Data":"ca2fb850efe761fb943bace4954dd9e17d24ae3c8c361ef27a9f6c6446bf8fcf"}
Sep 16 14:31:41.550110 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:41.550061    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd" podStartSLOduration=1.550044382 podStartE2EDuration="1.550044382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:31:41.547973669 +0000 UTC m=+1632.035983744" watchObservedRunningTime="2026-09-16 14:31:41.550044382 +0000 UTC m=+1632.038054457"
Sep 16 14:31:44.868804 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:44.868771    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:31:51.089779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:51.089742    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:51.090424 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:51.089790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:51.102238 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:51.102213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:51.588981 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:51.588952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:31:55.981951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:31:55.981914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:02.688244 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:02.688199    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:02.791435 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:02.791393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:02.919670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:02.919631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:07.140119 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:07.140080    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:32:13.676800 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:13.676765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:14.100615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:14.100578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:19.712744 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:19.712712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:32:30.311482 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:30.311450    2580 scope.go:117] "RemoveContainer" containerID="b8b48ec89d86973e4d2b384e8033e32c328d04603d42d693b0b37aafa293ad2f"
Sep 16 14:32:31.492160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:31.492095    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:32:34.880769 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:34.880733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:35.000288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:35.000251    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:32:36.229873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:36.229838    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"]
Sep 16 14:32:36.327700 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:36.327667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:37.225149 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:37.225106    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"]
Sep 16 14:32:37.251377 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:37.251347    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:37.754525 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:37.754491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:38.171560 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:38.171478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:38.681522 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:38.681488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:41.251389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:41.251347    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:42.216690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:42.216652    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:42.439515 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:42.439482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:42.939121 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:42.939082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:43.124071 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.124031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:43.224739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.224693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:43.477711 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.477609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:43.855414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.855335    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"]
Sep 16 14:32:43.859059 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.859040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.863022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.862998    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7b98f7-openshift-default-dockercfg-r9fkm\""
Sep 16 14:32:43.892898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.892868    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"]
Sep 16 14:32:43.921291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr57h\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-kube-api-access-sr57h\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-envoy\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-socket\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-token\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-credential-socket\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-podinfo\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-certs\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-data\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.921725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.921548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istiod-ca-cert\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:43.931596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:43.931567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:44.022372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-data\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istiod-ca-cert\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr57h\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-kube-api-access-sr57h\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-envoy\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-socket\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-token\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-credential-socket\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-podinfo\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-certs\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.022807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-data\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.023034 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.022989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-socket\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.023081 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.023070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-credential-socket\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.023117 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.023096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-certs\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.023227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.023202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istiod-ca-cert\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.024928 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.024890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-envoy\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.025673 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.025652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-podinfo\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.038972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.038944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr57h\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-kube-api-access-sr57h\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.041067 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.041045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-token\") pod \"e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") " pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.043132 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.043107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:44.171127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.171040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:44.466315 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.466288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"]
Sep 16 14:32:44.468061 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:32:44.468032    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07942cde_c97c_4436_bcd0_a6a4d054d7c1.slice/crio-e462a2b6eeb55dfafc56e5275b98a4666fff732b96b704dd7f4c47cda974edff WatchSource:0}: Error finding container e462a2b6eeb55dfafc56e5275b98a4666fff732b96b704dd7f4c47cda974edff: Status 404 returned error can't find the container with id e462a2b6eeb55dfafc56e5275b98a4666fff732b96b704dd7f4c47cda974edff
Sep 16 14:32:44.470327 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.470294    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:32:44.470406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.470370    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:32:44.470406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.470397    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:32:44.850457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.850355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:44.871483 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.871452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" event={"ID":"07942cde-c97c-4436-bcd0-a6a4d054d7c1","Type":"ContainerStarted","Data":"45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168"}
Sep 16 14:32:44.871678 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.871510    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" event={"ID":"07942cde-c97c-4436-bcd0-a6a4d054d7c1","Type":"ContainerStarted","Data":"e462a2b6eeb55dfafc56e5275b98a4666fff732b96b704dd7f4c47cda974edff"}
Sep 16 14:32:44.923606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:44.923544    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podStartSLOduration=1.9235234110000001 podStartE2EDuration="1.923523411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:44.921909163 +0000 UTC m=+1695.409919258" watchObservedRunningTime="2026-09-16 14:32:44.923523411 +0000 UTC m=+1695.411533488"
Sep 16 14:32:45.000721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:45.000689    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:32:45.171609 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:45.171517    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:45.176830 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:45.176804    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:45.739861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:45.739823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:45.877524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:45.877488    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:45.878479 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:45.878459    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:32:46.878294 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:46.878254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:46.993123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:46.993082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:47.117934 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:47.117898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:47.241513 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:47.241481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:47.530013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:47.529922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:48.055926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.055887    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:48.154470 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.154434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:48.325758 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.325657    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"]
Sep 16 14:32:48.329475 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.329456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.332728 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.332706    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b77fde-dockercfg-55c8t\""
Sep 16 14:32:48.332892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.332862    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b77fde\""
Sep 16 14:32:48.354384 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.354349    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"]
Sep 16 14:32:48.471736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.471696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c963f4c2-fab1-4b97-b519-3c7a12008de5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.471917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.471750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grg6n\" (UniqueName: \"kubernetes.io/projected/c963f4c2-fab1-4b97-b519-3c7a12008de5-kube-api-access-grg6n\") pod \"payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.497657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.497624    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:32:48.573188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.573131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grg6n\" (UniqueName: \"kubernetes.io/projected/c963f4c2-fab1-4b97-b519-3c7a12008de5-kube-api-access-grg6n\") pod \"payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.573422 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.573402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c963f4c2-fab1-4b97-b519-3c7a12008de5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.574096 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.574073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c963f4c2-fab1-4b97-b519-3c7a12008de5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.584017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.583945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grg6n\" (UniqueName: \"kubernetes.io/projected/c963f4c2-fab1-4b97-b519-3c7a12008de5-kube-api-access-grg6n\") pod \"payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.641039 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.640987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:48.937943 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.937913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"]
Sep 16 14:32:48.940330 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:32:48.940297    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc963f4c2_fab1_4b97_b519_3c7a12008de5.slice/crio-6be1c275e1c2ba9c29b2737aab24e014e36409e9739d924079a70c74ff451444 WatchSource:0}: Error finding container 6be1c275e1c2ba9c29b2737aab24e014e36409e9739d924079a70c74ff451444: Status 404 returned error can't find the container with id 6be1c275e1c2ba9c29b2737aab24e014e36409e9739d924079a70c74ff451444
Sep 16 14:32:48.977011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:48.976977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:49.073340 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.073297    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:49.477788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.477758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:49.825798 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.825708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:49.901569 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.901531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh" event={"ID":"c963f4c2-fab1-4b97-b519-3c7a12008de5","Type":"ContainerStarted","Data":"3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69"}
Sep 16 14:32:49.901786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.901598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh" event={"ID":"c963f4c2-fab1-4b97-b519-3c7a12008de5","Type":"ContainerStarted","Data":"6be1c275e1c2ba9c29b2737aab24e014e36409e9739d924079a70c74ff451444"}
Sep 16 14:32:49.901786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.901730    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:32:49.928794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.928737    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh" podStartSLOduration=1.9287190779999999 podStartE2EDuration="1.928719078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:49.928400014 +0000 UTC m=+1700.416410092" watchObservedRunningTime="2026-09-16 14:32:49.928719078 +0000 UTC m=+1700.416729154"
Sep 16 14:32:49.939790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:49.939759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:50.008443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.008416    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"]
Sep 16 14:32:50.012090 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.012072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.014888 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.014866    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7b98f7-dockercfg-w55kd\""
Sep 16 14:32:50.015255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.015237    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7b98f7\""
Sep 16 14:32:50.020287 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.019565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"]
Sep 16 14:32:50.091124 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.091018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qpnv\" (UniqueName: \"kubernetes.io/projected/67c4cb09-263d-4da4-913a-c6156fe0df60-kube-api-access-5qpnv\") pod \"payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.091575 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.091263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c4cb09-263d-4da4-913a-c6156fe0df60-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.192868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.192829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c4cb09-263d-4da4-913a-c6156fe0df60-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.193099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.192888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qpnv\" (UniqueName: \"kubernetes.io/projected/67c4cb09-263d-4da4-913a-c6156fe0df60-kube-api-access-5qpnv\") pod \"payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.193705 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.193676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c4cb09-263d-4da4-913a-c6156fe0df60-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.203191 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.203162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qpnv\" (UniqueName: \"kubernetes.io/projected/67c4cb09-263d-4da4-913a-c6156fe0df60-kube-api-access-5qpnv\") pod \"payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.324165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.324101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.420995 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.420959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:50.613423 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.613318    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"]
Sep 16 14:32:50.616255 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:32:50.616212    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c4cb09_263d_4da4_913a_c6156fe0df60.slice/crio-b93464ddfa4aabc8e58861b8eb48df9a7d0ea218ee7e92af5287ea2413df83f7 WatchSource:0}: Error finding container b93464ddfa4aabc8e58861b8eb48df9a7d0ea218ee7e92af5287ea2413df83f7: Status 404 returned error can't find the container with id b93464ddfa4aabc8e58861b8eb48df9a7d0ea218ee7e92af5287ea2413df83f7
Sep 16 14:32:50.908607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.908510    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm" event={"ID":"67c4cb09-263d-4da4-913a-c6156fe0df60","Type":"ContainerStarted","Data":"73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7"}
Sep 16 14:32:50.908607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.908568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm" event={"ID":"67c4cb09-263d-4da4-913a-c6156fe0df60","Type":"ContainerStarted","Data":"b93464ddfa4aabc8e58861b8eb48df9a7d0ea218ee7e92af5287ea2413df83f7"}
Sep 16 14:32:50.909013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.908812    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:32:50.932107 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:50.932045    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm" podStartSLOduration=1.932027187 podStartE2EDuration="1.932027187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:50.931114959 +0000 UTC m=+1701.419125037" watchObservedRunningTime="2026-09-16 14:32:50.932027187 +0000 UTC m=+1701.420037288"
Sep 16 14:32:51.248918 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.248876    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp" podUID="ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" containerName="main" containerID="cri-o://80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e" gracePeriod=45
Sep 16 14:32:51.506887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.506820    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:32:51.563458 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.563421    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:51.608369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608331    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tmp-dir\") pod \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") "
Sep 16 14:32:51.608547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608389    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-home\") pod \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") "
Sep 16 14:32:51.608547 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608455    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-model-cache\") pod \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") "
Sep 16 14:32:51.608648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-dshm\") pod \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") "
Sep 16 14:32:51.608648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608601    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tls-certs\") pod \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") "
Sep 16 14:32:51.608648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608635    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdrld\" (UniqueName: \"kubernetes.io/projected/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-kube-api-access-bdrld\") pod \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\" (UID: \"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4\") "
Sep 16 14:32:51.608806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608667    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" (UID: "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:51.608806 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608702    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-home" (OuterVolumeSpecName: "home") pod "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" (UID: "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:51.608975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.608819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" (UID: "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:51.609047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.609028    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:51.609100 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.609054    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:51.609100 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.609068    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:51.610845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.610820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" (UID: "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:32:51.610845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.610827    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-dshm" (OuterVolumeSpecName: "dshm") pod "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" (UID: "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:51.610998 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.610848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-kube-api-access-bdrld" (OuterVolumeSpecName: "kube-api-access-bdrld") pod "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" (UID: "ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4"). InnerVolumeSpecName "kube-api-access-bdrld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:32:51.710653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.710609    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:51.710653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.710644    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:51.710653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.710657    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdrld\" (UniqueName: \"kubernetes.io/projected/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4-kube-api-access-bdrld\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:51.916320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.916220    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" containerID="80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e" exitCode=0
Sep 16 14:32:51.916320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.916274    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"
Sep 16 14:32:51.916320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.916301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp" event={"ID":"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4","Type":"ContainerDied","Data":"80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e"}
Sep 16 14:32:51.916596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.916345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp" event={"ID":"ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4","Type":"ContainerDied","Data":"07f7b754faf8633a43bce54367ea130f1d4ab64718e9db4364202127aca74404"}
Sep 16 14:32:51.916596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.916365    2580 scope.go:117] "RemoveContainer" containerID="80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e"
Sep 16 14:32:51.926654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.926634    2580 scope.go:117] "RemoveContainer" containerID="80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e"
Sep 16 14:32:51.926991 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:32:51.926970    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e\": container with ID starting with 80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e not found: ID does not exist" containerID="80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e"
Sep 16 14:32:51.927047 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.927001    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e"} err="failed to get container status \"80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e\": rpc error: code = NotFound desc = could not find container \"80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e\": container with ID starting with 80f66d8b752de86355d53e19430e0bbd7b7c9c41174dc806790cc16b5cc9049e not found: ID does not exist"
Sep 16 14:32:51.952706 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.952662    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"]
Sep 16 14:32:51.957162 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:51.957121    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-50b57c/rate-test-model-50b57c-kserve-7c4958b987-tpccp"]
Sep 16 14:32:52.007390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.007351    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" path="/var/lib/kubelet/pods/ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4/volumes"
Sep 16 14:32:52.244345 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.244301    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd" podUID="b3057ab4-9bf2-493e-8e63-3acb575cbc46" containerName="main" containerID="cri-o://b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e" gracePeriod=45
Sep 16 14:32:52.503854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.503782    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:32:52.565367 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.565331    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:52.620448 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620412    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tmp-dir\") pod \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") "
Sep 16 14:32:52.620654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620525    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tls-certs\") pod \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") "
Sep 16 14:32:52.620654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-model-cache\") pod \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") "
Sep 16 14:32:52.620654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620573    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-home\") pod \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") "
Sep 16 14:32:52.620654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620589    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-dshm\") pod \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") "
Sep 16 14:32:52.620654 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620641    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2j9m\" (UniqueName: \"kubernetes.io/projected/b3057ab4-9bf2-493e-8e63-3acb575cbc46-kube-api-access-k2j9m\") pod \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\" (UID: \"b3057ab4-9bf2-493e-8e63-3acb575cbc46\") "
Sep 16 14:32:52.620927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620772    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b3057ab4-9bf2-493e-8e63-3acb575cbc46" (UID: "b3057ab4-9bf2-493e-8e63-3acb575cbc46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:52.620927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.620860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-home" (OuterVolumeSpecName: "home") pod "b3057ab4-9bf2-493e-8e63-3acb575cbc46" (UID: "b3057ab4-9bf2-493e-8e63-3acb575cbc46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:52.621064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.621039    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-model-cache" (OuterVolumeSpecName: "model-cache") pod "b3057ab4-9bf2-493e-8e63-3acb575cbc46" (UID: "b3057ab4-9bf2-493e-8e63-3acb575cbc46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:52.621118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.621066    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:52.621118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.621087    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:52.622793 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.622762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b3057ab4-9bf2-493e-8e63-3acb575cbc46" (UID: "b3057ab4-9bf2-493e-8e63-3acb575cbc46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:32:52.622793 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.622773    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-dshm" (OuterVolumeSpecName: "dshm") pod "b3057ab4-9bf2-493e-8e63-3acb575cbc46" (UID: "b3057ab4-9bf2-493e-8e63-3acb575cbc46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:32:52.622945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.622864    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3057ab4-9bf2-493e-8e63-3acb575cbc46-kube-api-access-k2j9m" (OuterVolumeSpecName: "kube-api-access-k2j9m") pod "b3057ab4-9bf2-493e-8e63-3acb575cbc46" (UID: "b3057ab4-9bf2-493e-8e63-3acb575cbc46"). InnerVolumeSpecName "kube-api-access-k2j9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:32:52.722434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.722375    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2j9m\" (UniqueName: \"kubernetes.io/projected/b3057ab4-9bf2-493e-8e63-3acb575cbc46-kube-api-access-k2j9m\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:52.722434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.722428    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3057ab4-9bf2-493e-8e63-3acb575cbc46-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:52.722434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.722444    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:52.722694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.722456    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3057ab4-9bf2-493e-8e63-3acb575cbc46-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:32:52.923991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.923898    2580 generic.go:356] "Generic (PLEG): container finished" podID="b3057ab4-9bf2-493e-8e63-3acb575cbc46" containerID="b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e" exitCode=0
Sep 16 14:32:52.923991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.923943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"
Sep 16 14:32:52.923991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.923980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd" event={"ID":"b3057ab4-9bf2-493e-8e63-3acb575cbc46","Type":"ContainerDied","Data":"b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e"}
Sep 16 14:32:52.924284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.924026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd" event={"ID":"b3057ab4-9bf2-493e-8e63-3acb575cbc46","Type":"ContainerDied","Data":"ca2fb850efe761fb943bace4954dd9e17d24ae3c8c361ef27a9f6c6446bf8fcf"}
Sep 16 14:32:52.924284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.924050    2580 scope.go:117] "RemoveContainer" containerID="b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e"
Sep 16 14:32:52.934103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.934078    2580 scope.go:117] "RemoveContainer" containerID="b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e"
Sep 16 14:32:52.934469 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:32:52.934450    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e\": container with ID starting with b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e not found: ID does not exist" containerID="b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e"
Sep 16 14:32:52.934528 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.934479    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e"} err="failed to get container status \"b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e\": rpc error: code = NotFound desc = could not find container \"b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e\": container with ID starting with b1e60adb032d7767c7dfe318828072e6efc6d53d24ae915e1cb053b1a68f4e8e not found: ID does not exist"
Sep 16 14:32:52.947339 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.947300    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"]
Sep 16 14:32:52.955295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:52.955266    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f72402/rate-test-model-f72402-kserve-ff7f6f559-vjwtd"]
Sep 16 14:32:53.396268 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:53.396230    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:54.006838 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:54.006803    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3057ab4-9bf2-493e-8e63-3acb575cbc46" path="/var/lib/kubelet/pods/b3057ab4-9bf2-493e-8e63-3acb575cbc46/volumes"
Sep 16 14:32:54.450179 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:54.450069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:32:55.376380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:32:55.376349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:00.911440 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:00.911390    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:33:01.919390 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:01.919351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:33:07.593200 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:07.593130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:33:08.734053 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:08.734010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:09.640034 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:09.639931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:09.812153 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:09.812114    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:10.038179 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:10.038130    2580 generic.go:356] "Generic (PLEG): container finished" podID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerID="45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168" exitCode=0
Sep 16 14:33:10.038369 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:10.038204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" event={"ID":"07942cde-c97c-4436-bcd0-a6a4d054d7c1","Type":"ContainerDied","Data":"45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168"}
Sep 16 14:33:10.038656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:10.038642    2580 scope.go:117] "RemoveContainer" containerID="45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168"
Sep 16 14:33:10.039916 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:10.039883    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:10.040030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:10.039973    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:10.040030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:10.040023    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:10.799365 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:10.799320    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:11.046837 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:11.046798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" event={"ID":"07942cde-c97c-4436-bcd0-a6a4d054d7c1","Type":"ContainerStarted","Data":"b25cb4877046b5db84c34092254410fa39e7688d494d28570d7d3f0fedb04a42"}
Sep 16 14:33:11.172263 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:11.172129    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:33:11.173398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:11.173368    2580 patch_prober.go:28] interesting pod/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 16 14:33:11.173524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:11.173425    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 16 14:33:11.708466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:11.708414    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:12.093420 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.093331    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:12.172479 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.172436    2580 patch_prober.go:28] interesting pod/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 16 14:33:12.172637 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.172503    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 16 14:33:12.418994 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.418884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:12.741341 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.741303    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"]
Sep 16 14:33:12.741807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.741792    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3057ab4-9bf2-493e-8e63-3acb575cbc46" containerName="main"
Sep 16 14:33:12.741807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.741807    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3057ab4-9bf2-493e-8e63-3acb575cbc46" containerName="main"
Sep 16 14:33:12.741908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.741821    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" containerName="main"
Sep 16 14:33:12.741908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.741827    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" containerName="main"
Sep 16 14:33:12.741908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.741906    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3057ab4-9bf2-493e-8e63-3acb575cbc46" containerName="main"
Sep 16 14:33:12.742066 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.741916    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad17a8a4-4d80-4fa2-b5d1-e660816f9fa4" containerName="main"
Sep 16 14:33:12.744412 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.744384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.747675 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.747651    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-7eeba8-openshift-default-dockercfg-8jnzb\""
Sep 16 14:33:12.758572 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.758539    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"]
Sep 16 14:33:12.844311 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-istio-token\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-data\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b28b172-88cb-4d69-8598-29720071a0a9-istiod-ca-cert\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b28b172-88cb-4d69-8598-29720071a0a9-istio-podinfo\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-envoy\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-certs\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w998z\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-kube-api-access-w998z\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-socket\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.844951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.844804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-credential-socket\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-data\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b28b172-88cb-4d69-8598-29720071a0a9-istiod-ca-cert\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b28b172-88cb-4d69-8598-29720071a0a9-istio-podinfo\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-envoy\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-certs\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w998z\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-kube-api-access-w998z\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-socket\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-credential-socket\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.946782 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-istio-token\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.947331 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-data\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.947331 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.946935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-certs\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.947331 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.947199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-socket\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.947331 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.947249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-credential-socket\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.947510 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.947398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b28b172-88cb-4d69-8598-29720071a0a9-istiod-ca-cert\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.949073 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.949049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-envoy\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.949372 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.949352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b28b172-88cb-4d69-8598-29720071a0a9-istio-podinfo\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.970053 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.970017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-istio-token\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:12.977107 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:12.977075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w998z\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-kube-api-access-w998z\") pod \"e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") " pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:13.056502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.056402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:13.172398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.172313    2580 patch_prober.go:28] interesting pod/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 16 14:33:13.172398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.172388    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 16 14:33:13.365719 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.365613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:13.372874 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.372838    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"]
Sep 16 14:33:13.374632 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:33:13.374606    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b28b172_88cb_4d69_8598_29720071a0a9.slice/crio-028387bd41db41f7a132f51141311207e08c04aaaab31be0e02fa9d6d40899ee WatchSource:0}: Error finding container 028387bd41db41f7a132f51141311207e08c04aaaab31be0e02fa9d6d40899ee: Status 404 returned error can't find the container with id 028387bd41db41f7a132f51141311207e08c04aaaab31be0e02fa9d6d40899ee
Sep 16 14:33:13.377312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.377272    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:13.377414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.377352    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:13.377414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.377385    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:13.987693 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:13.987649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:14.074983 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:14.074942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" event={"ID":"0b28b172-88cb-4d69-8598-29720071a0a9","Type":"ContainerStarted","Data":"e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a"}
Sep 16 14:33:14.075227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:14.075039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" event={"ID":"0b28b172-88cb-4d69-8598-29720071a0a9","Type":"ContainerStarted","Data":"028387bd41db41f7a132f51141311207e08c04aaaab31be0e02fa9d6d40899ee"}
Sep 16 14:33:14.095788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:14.095748    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:14.104969 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:14.104893    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" podStartSLOduration=2.10486882 podStartE2EDuration="2.10486882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:14.101913492 +0000 UTC m=+1724.589923578" watchObservedRunningTime="2026-09-16 14:33:14.10486882 +0000 UTC m=+1724.592878898"
Sep 16 14:33:14.172434 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:14.172390    2580 patch_prober.go:28] interesting pod/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 16 14:33:14.172828 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:14.172455    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 16 14:33:14.930938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:14.930896    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:15.057011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.056971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:15.062606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.062579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:15.080509 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.080484    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:15.081503 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.081482    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:15.171812 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.171772    2580 patch_prober.go:28] interesting pod/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 16 14:33:15.171992 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.171846    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 16 14:33:15.877882 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.877846    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:33:15.939807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:15.939766    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:16.119163 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:16.118866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:16.172118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:16.172027    2580 patch_prober.go:28] interesting pod/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Sep 16 14:33:16.172118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:16.172105    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Sep 16 14:33:16.290904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:16.290869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:17.040085 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.040046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:17.139478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.139422    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:17.176758 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.176726    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:33:17.177997 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.177971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:33:17.580069 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.580024    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:17.868106 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.868004    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"]
Sep 16 14:33:17.871004 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.870979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:17.874581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.874558    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-927f98-openshift-default-dockercfg-lh7m4\""
Sep 16 14:33:17.884218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.884188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"]
Sep 16 14:33:17.925500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:17.925448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:18.000171 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-envoy\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sspfk\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-kube-api-access-sspfk\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509b7b13-3e3a-45c3-856b-8bf53a9299df-istiod-ca-cert\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-certs\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000486 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-credential-socket\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000486 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-socket\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-data\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000668 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-podinfo\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.000708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.000698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-token\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.024273 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.024233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:18.102183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-data\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-podinfo\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-token\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-envoy\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sspfk\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-kube-api-access-sspfk\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509b7b13-3e3a-45c3-856b-8bf53a9299df-istiod-ca-cert\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-certs\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-credential-socket\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-socket\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.102666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-data\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.103054 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-certs\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.103054 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-credential-socket\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.103054 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.102838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-socket\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.103418 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.103399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509b7b13-3e3a-45c3-856b-8bf53a9299df-istiod-ca-cert\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.104619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.104601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-envoy\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.105022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.105001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-podinfo\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.116504 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.116468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-token\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.116927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.116905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sspfk\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-kube-api-access-sspfk\") pod \"e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") " pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.149233 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.149106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:18.183967 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.183918    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:18.466004 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.465979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"]
Sep 16 14:33:18.467971 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:33:18.467942    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod509b7b13_3e3a_45c3_856b_8bf53a9299df.slice/crio-94c60300232677d3df23fb459dcac08a0b174cdf69538d41bdc48e1cd377b80e WatchSource:0}: Error finding container 94c60300232677d3df23fb459dcac08a0b174cdf69538d41bdc48e1cd377b80e: Status 404 returned error can't find the container with id 94c60300232677d3df23fb459dcac08a0b174cdf69538d41bdc48e1cd377b80e
Sep 16 14:33:18.470265 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.470227    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:18.470375 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.470316    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:18.470375 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:18.470359    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:19.105211 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.105162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" event={"ID":"509b7b13-3e3a-45c3-856b-8bf53a9299df","Type":"ContainerStarted","Data":"5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221"}
Sep 16 14:33:19.105805 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.105246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" event={"ID":"509b7b13-3e3a-45c3-856b-8bf53a9299df","Type":"ContainerStarted","Data":"94c60300232677d3df23fb459dcac08a0b174cdf69538d41bdc48e1cd377b80e"}
Sep 16 14:33:19.126634 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.126555    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" podStartSLOduration=2.126538259 podStartE2EDuration="2.126538259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:19.125688597 +0000 UTC m=+1729.613698673" watchObservedRunningTime="2026-09-16 14:33:19.126538259 +0000 UTC m=+1729.614548336"
Sep 16 14:33:19.184130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.184096    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:19.191451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.191402    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:19.241732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.241696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:19.343567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.343533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:19.440514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.440404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:19.769439 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.769382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:19.869546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.869511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:19.958863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.958831    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"]
Sep 16 14:33:19.961561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.961544    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:19.964212 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.964188    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-7eeba8\""
Sep 16 14:33:19.964343 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.964276    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-7eeba8-dockercfg-vk8s4\""
Sep 16 14:33:19.973576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:19.973551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"]
Sep 16 14:33:20.025545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.025439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g897\" (UniqueName: \"kubernetes.io/projected/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-kube-api-access-4g897\") pod \"payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:20.025731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.025665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:20.064754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.064722    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"]
Sep 16 14:33:20.067599 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.067582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.082487 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.082456    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"]
Sep 16 14:33:20.112380 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.112345    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:20.113344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.113323    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:20.127094 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.127062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g897\" (UniqueName: \"kubernetes.io/projected/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-kube-api-access-4g897\") pod \"payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:20.127288 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.127118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rrnw\" (UniqueName: \"kubernetes.io/projected/72125b19-c7ae-4f80-8d62-b0cdd93cab30-kube-api-access-8rrnw\") pod \"payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.127361 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.127291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:20.127361 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.127321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72125b19-c7ae-4f80-8d62-b0cdd93cab30-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.128011 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.127988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:20.137810 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.137784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g897\" (UniqueName: \"kubernetes.io/projected/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-kube-api-access-4g897\") pod \"payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:20.228075 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.228038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rrnw\" (UniqueName: \"kubernetes.io/projected/72125b19-c7ae-4f80-8d62-b0cdd93cab30-kube-api-access-8rrnw\") pod \"payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.228280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.228182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72125b19-c7ae-4f80-8d62-b0cdd93cab30-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.228885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.228862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72125b19-c7ae-4f80-8d62-b0cdd93cab30-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.238590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.238567    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rrnw\" (UniqueName: \"kubernetes.io/projected/72125b19-c7ae-4f80-8d62-b0cdd93cab30-kube-api-access-8rrnw\") pod \"payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.273458 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.273415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:20.377811 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.377768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:20.463546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.463510    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:20.549517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.549491    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"]
Sep 16 14:33:20.551625 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:33:20.551597    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9ac34b_2021_4b54_8af7_50dd483d1e0f.slice/crio-06a70ab2a228a7ebd04f961ad8015bcca4863171cc838ee09e563b98d1b45121 WatchSource:0}: Error finding container 06a70ab2a228a7ebd04f961ad8015bcca4863171cc838ee09e563b98d1b45121: Status 404 returned error can't find the container with id 06a70ab2a228a7ebd04f961ad8015bcca4863171cc838ee09e563b98d1b45121
Sep 16 14:33:20.659672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:20.659645    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"]
Sep 16 14:33:20.661559 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:33:20.661532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72125b19_c7ae_4f80_8d62_b0cdd93cab30.slice/crio-5e665e8dd837b6af3fb08fa0a98ee96d2d2d0cdb736158e7092f246cc7fbb1a9 WatchSource:0}: Error finding container 5e665e8dd837b6af3fb08fa0a98ee96d2d2d0cdb736158e7092f246cc7fbb1a9: Status 404 returned error can't find the container with id 5e665e8dd837b6af3fb08fa0a98ee96d2d2d0cdb736158e7092f246cc7fbb1a9
Sep 16 14:33:21.120594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.120111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd" event={"ID":"ae9ac34b-2021-4b54-8af7-50dd483d1e0f","Type":"ContainerStarted","Data":"c23ccacc34aa314b9294c06bf9482a01cc458be6f820b4fd973b8d9eb71a8780"}
Sep 16 14:33:21.120594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.120203    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:21.120594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.120228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd" event={"ID":"ae9ac34b-2021-4b54-8af7-50dd483d1e0f","Type":"ContainerStarted","Data":"06a70ab2a228a7ebd04f961ad8015bcca4863171cc838ee09e563b98d1b45121"}
Sep 16 14:33:21.122865 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.122821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr" event={"ID":"72125b19-c7ae-4f80-8d62-b0cdd93cab30","Type":"ContainerStarted","Data":"0e9b5fc347573a2b35f9a94c4a83c1b6a1611e6991612393665b9521b9bdf1fd"}
Sep 16 14:33:21.123020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.122870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr" event={"ID":"72125b19-c7ae-4f80-8d62-b0cdd93cab30","Type":"ContainerStarted","Data":"5e665e8dd837b6af3fb08fa0a98ee96d2d2d0cdb736158e7092f246cc7fbb1a9"}
Sep 16 14:33:21.123020 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.122901    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:21.153276 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.153212    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd" podStartSLOduration=2.15318678 podStartE2EDuration="2.15318678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:21.150094475 +0000 UTC m=+1731.638104555" watchObservedRunningTime="2026-09-16 14:33:21.15318678 +0000 UTC m=+1731.641196843"
Sep 16 14:33:21.165389 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.165353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:21.178910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.178852    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr" podStartSLOduration=1.178835363 podStartE2EDuration="1.178835363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:21.177553327 +0000 UTC m=+1731.665563410" watchObservedRunningTime="2026-09-16 14:33:21.178835363 +0000 UTC m=+1731.666845439"
Sep 16 14:33:21.271583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.271539    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:21.366414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:21.366375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:22.422955 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:22.422924    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:23.410165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:23.410069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:23.506270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:23.506238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:23.743989 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:23.743955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:24.084566 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:24.084468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:24.186383 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:24.186336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:24.619490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:24.619453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:25.968195 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:25.968155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:26.777746 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:26.777710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:27.715014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:27.714978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:28.188516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:28.188416    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:33:29.153864 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:29.153823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:29.734048 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:29.734009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:30.334388 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:30.334344    2580 scope.go:117] "RemoveContainer" containerID="c26437557c6add19c53b09cdf012732d5794b2051da172319ae091ac5c7d4e92"
Sep 16 14:33:31.060451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:31.060407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:32.133069 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:32.133037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:33:32.134096 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:32.134074    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:33:37.000837 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:37.000791    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:33:39.243354 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.243314    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b28b172-88cb-4d69-8598-29720071a0a9" containerID="e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a" exitCode=0
Sep 16 14:33:39.243863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.243381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" event={"ID":"0b28b172-88cb-4d69-8598-29720071a0a9","Type":"ContainerDied","Data":"e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a"}
Sep 16 14:33:39.243863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.243814    2580 scope.go:117] "RemoveContainer" containerID="e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a"
Sep 16 14:33:39.245289 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.245254    2580 generic.go:356] "Generic (PLEG): container finished" podID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerID="5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221" exitCode=0
Sep 16 14:33:39.245425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.245303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" event={"ID":"509b7b13-3e3a-45c3-856b-8bf53a9299df","Type":"ContainerDied","Data":"5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221"}
Sep 16 14:33:39.245425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.245317    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:39.245425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.245399    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:39.245540 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.245435    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:39.245721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.245709    2580 scope.go:117] "RemoveContainer" containerID="5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221"
Sep 16 14:33:39.246942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.246911    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:39.247051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.246976    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:39.247051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:39.247011    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:33:40.255751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:40.255710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" event={"ID":"0b28b172-88cb-4d69-8598-29720071a0a9","Type":"ContainerStarted","Data":"341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec"}
Sep 16 14:33:40.257754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:40.257725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" event={"ID":"509b7b13-3e3a-45c3-856b-8bf53a9299df","Type":"ContainerStarted","Data":"b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44"}
Sep 16 14:33:40.463397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:40.463344    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:41.057535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:41.057496    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:41.058675 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:41.058644    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 16 14:33:41.058736 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:41.058707    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 16 14:33:41.184513 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:41.184480    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:41.184778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:41.184755    2580 patch_prober.go:28] interesting pod/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Sep 16 14:33:41.184875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:41.184804    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Sep 16 14:33:41.992443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:41.992407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:42.056843 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:42.056803    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 16 14:33:42.057037 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:42.056888    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 16 14:33:42.184656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:42.184614    2580 patch_prober.go:28] interesting pod/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Sep 16 14:33:42.184885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:42.184671    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Sep 16 14:33:42.902703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:42.902651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:43.057445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:43.057406    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 16 14:33:43.057827 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:43.057481    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 16 14:33:43.184652 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:43.184458    2580 patch_prober.go:28] interesting pod/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Sep 16 14:33:43.184652 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:43.184515    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Sep 16 14:33:43.840157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:43.840105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:44.057149 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:44.057097    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 16 14:33:44.057337 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:44.057193    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 16 14:33:44.185324 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:44.185229    2580 patch_prober.go:28] interesting pod/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Sep 16 14:33:44.185324 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:44.185289    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Sep 16 14:33:44.775958 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:44.775923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:45.057084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:45.056980    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 16 14:33:45.057084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:45.057054    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 16 14:33:45.080606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:45.080567    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:45.185391 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:45.185352    2580 patch_prober.go:28] interesting pod/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Sep 16 14:33:45.185798 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:45.185425    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Sep 16 14:33:45.959907 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:45.959865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:46.061232 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:46.061203    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:46.062439 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:46.062414    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:33:46.188722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:46.188693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:46.189104 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:46.188993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:46.189740 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:46.189720    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:33:46.206113 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:46.206083    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:46.326168 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:46.326066    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:47.124699 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:47.122402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:48.212455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:48.212423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:49.273761 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.273724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:49.430086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.430052    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"]
Sep 16 14:33:49.433543 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.433524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"
Sep 16 14:33:49.437535 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.437514    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:33:49.437704 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.437688    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:33:49.445988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.445963    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"]
Sep 16 14:33:49.556191 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.556051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cfe4f51-d748-4bbb-8c87-b81728da1b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb\" (UID: \"3cfe4f51-d748-4bbb-8c87-b81728da1b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"
Sep 16 14:33:49.657488 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.657447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cfe4f51-d748-4bbb-8c87-b81728da1b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb\" (UID: \"3cfe4f51-d748-4bbb-8c87-b81728da1b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"
Sep 16 14:33:49.658105 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.658083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cfe4f51-d748-4bbb-8c87-b81728da1b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb\" (UID: \"3cfe4f51-d748-4bbb-8c87-b81728da1b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"
Sep 16 14:33:49.744311 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:49.744271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"
Sep 16 14:33:50.038935 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:50.038911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"]
Sep 16 14:33:50.040771 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:33:50.040737    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfe4f51_d748_4bbb_8c87_b81728da1b27.slice/crio-50190135b85565a366fefede988d0145cacd1be28b3daa9658e1d7f71349605f WatchSource:0}: Error finding container 50190135b85565a366fefede988d0145cacd1be28b3daa9658e1d7f71349605f: Status 404 returned error can't find the container with id 50190135b85565a366fefede988d0145cacd1be28b3daa9658e1d7f71349605f
Sep 16 14:33:50.165734 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:50.165693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:50.326348 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:50.326314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb" event={"ID":"3cfe4f51-d748-4bbb-8c87-b81728da1b27","Type":"ContainerStarted","Data":"3429657d31991ee08d631a4cb982f89c8e955e4917b7bd8dda79b1d74e744f6c"}
Sep 16 14:33:50.326755 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:50.326368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb" event={"ID":"3cfe4f51-d748-4bbb-8c87-b81728da1b27","Type":"ContainerStarted","Data":"50190135b85565a366fefede988d0145cacd1be28b3daa9658e1d7f71349605f"}
Sep 16 14:33:50.345548 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:50.345491    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb" podStartSLOduration=1.345472659 podStartE2EDuration="1.345472659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:50.343403584 +0000 UTC m=+1760.831413662" watchObservedRunningTime="2026-09-16 14:33:50.345472659 +0000 UTC m=+1760.833482736"
Sep 16 14:33:51.225056 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:51.225022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:51.333247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:51.333210    2580 generic.go:356] "Generic (PLEG): container finished" podID="3cfe4f51-d748-4bbb-8c87-b81728da1b27" containerID="3429657d31991ee08d631a4cb982f89c8e955e4917b7bd8dda79b1d74e744f6c" exitCode=0
Sep 16 14:33:51.333632 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:51.333296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb" event={"ID":"3cfe4f51-d748-4bbb-8c87-b81728da1b27","Type":"ContainerDied","Data":"3429657d31991ee08d631a4cb982f89c8e955e4917b7bd8dda79b1d74e744f6c"}
Sep 16 14:33:51.720469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:51.720441    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-28hd7_e27dd0dd-1f25-4d76-84de-32602bba121c/payload-processing/0.log"
Sep 16 14:33:51.859194 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:51.859160    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-28hd7_e27dd0dd-1f25-4d76-84de-32602bba121c/payload-processing/0.log"
Sep 16 14:33:52.002344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:52.002257    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr_72125b19-c7ae-4f80-8d62-b0cdd93cab30/payload-processing/0.log"
Sep 16 14:33:52.130378 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:52.130345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:52.475472 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:52.475449    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"
Sep 16 14:33:52.587953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:52.587921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cfe4f51-d748-4bbb-8c87-b81728da1b27-maas-api-service-ca\") pod \"3cfe4f51-d748-4bbb-8c87-b81728da1b27\" (UID: \"3cfe4f51-d748-4bbb-8c87-b81728da1b27\") "
Sep 16 14:33:52.588345 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:52.588324    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cfe4f51-d748-4bbb-8c87-b81728da1b27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3cfe4f51-d748-4bbb-8c87-b81728da1b27" (UID: "3cfe4f51-d748-4bbb-8c87-b81728da1b27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:33:52.689786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:52.689704    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cfe4f51-d748-4bbb-8c87-b81728da1b27-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:33:53.059771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:53.059728    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:53.347402 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:53.347326    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"
Sep 16 14:33:53.347402 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:53.347361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb" event={"ID":"3cfe4f51-d748-4bbb-8c87-b81728da1b27","Type":"ContainerDied","Data":"50190135b85565a366fefede988d0145cacd1be28b3daa9658e1d7f71349605f"}
Sep 16 14:33:53.347603 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:53.347409    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50190135b85565a366fefede988d0145cacd1be28b3daa9658e1d7f71349605f"
Sep 16 14:33:54.141904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:54.141865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:54.921045 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:54.920997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:55.917203 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:55.917163    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:56.674517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:56.674481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:58.000568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.000535    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:33:58.106844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.106808    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:58.177927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.177889    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"]
Sep 16 14:33:58.178474 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.178456    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cfe4f51-d748-4bbb-8c87-b81728da1b27" containerName="revoke-keys"
Sep 16 14:33:58.178474 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.178475    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfe4f51-d748-4bbb-8c87-b81728da1b27" containerName="revoke-keys"
Sep 16 14:33:58.178651 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.178638    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cfe4f51-d748-4bbb-8c87-b81728da1b27" containerName="revoke-keys"
Sep 16 14:33:58.180957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.180934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.184647 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.184618    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b77fde\"/\"ipp-route-b77fde-kserve-self-signed-certs\""
Sep 16 14:33:58.185778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.185758    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b77fde\"/\"kube-root-ca.crt\""
Sep 16 14:33:58.185890 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.185783    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b77fde\"/\"openshift-service-ca.crt\""
Sep 16 14:33:58.196060 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.196037    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b77fde\"/\"default-dockercfg-pj9rf\""
Sep 16 14:33:58.212844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.212814    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"]
Sep 16 14:33:58.309995 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.309904    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:33:58.355236 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.355196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcbh7\" (UniqueName: \"kubernetes.io/projected/acca96f1-fa3e-4350-92fa-aa3f074e75e0-kube-api-access-tcbh7\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.355425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.355307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tmp-dir\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.355425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.355373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-home\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.355425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.355388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-dshm\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.355425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.355405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-model-cache\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.355425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.355422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tls-certs\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.456674 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.456632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcbh7\" (UniqueName: \"kubernetes.io/projected/acca96f1-fa3e-4350-92fa-aa3f074e75e0-kube-api-access-tcbh7\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.456831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.456708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tmp-dir\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.456831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.456749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-home\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.456831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.456768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-dshm\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.456831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.456792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-model-cache\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.456831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.456830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tls-certs\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.457248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.457217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-home\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.457248 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.457241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tmp-dir\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.457384 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.457269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-model-cache\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.459191 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.459174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-dshm\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.459333 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.459316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tls-certs\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.468897 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.468870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcbh7\" (UniqueName: \"kubernetes.io/projected/acca96f1-fa3e-4350-92fa-aa3f074e75e0-kube-api-access-tcbh7\") pod \"ipp-route-b77fde-kserve-6b5f8997c5-j7spp\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") " pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.490290 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.490264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:33:58.777392 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:58.777361    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"]
Sep 16 14:33:58.779017 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:33:58.778986    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacca96f1_fa3e_4350_92fa_aa3f074e75e0.slice/crio-80a6e295fbb2bfacb87bbb8ae8b965206a8e841d9b1407419143c0d6c1e9d19c WatchSource:0}: Error finding container 80a6e295fbb2bfacb87bbb8ae8b965206a8e841d9b1407419143c0d6c1e9d19c: Status 404 returned error can't find the container with id 80a6e295fbb2bfacb87bbb8ae8b965206a8e841d9b1407419143c0d6c1e9d19c
Sep 16 14:33:59.390269 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:59.390233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp" event={"ID":"acca96f1-fa3e-4350-92fa-aa3f074e75e0","Type":"ContainerStarted","Data":"2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138"}
Sep 16 14:33:59.390269 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:59.390275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp" event={"ID":"acca96f1-fa3e-4350-92fa-aa3f074e75e0","Type":"ContainerStarted","Data":"80a6e295fbb2bfacb87bbb8ae8b965206a8e841d9b1407419143c0d6c1e9d19c"}
Sep 16 14:33:59.415231 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:59.415178    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp" podStartSLOduration=1.415160505 podStartE2EDuration="1.415160505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:59.411815553 +0000 UTC m=+1769.899825630" watchObservedRunningTime="2026-09-16 14:33:59.415160505 +0000 UTC m=+1769.903170582"
Sep 16 14:33:59.780378 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:33:59.780337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:04.043834 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:04.043794    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:34:05.465581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.465543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:05.514267 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.514231    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"]
Sep 16 14:34:05.514522 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.514492    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm" podUID="67c4cb09-263d-4da4-913a-c6156fe0df60" containerName="payload-pre-processing" containerID="cri-o://73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7" gracePeriod=30
Sep 16 14:34:05.567534 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.567500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:05.708175 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.707826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:05.794616 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.794585    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"]
Sep 16 14:34:05.803870 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.803845    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:34:05.806703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.806678    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7b98f7-xbfdb"]
Sep 16 14:34:05.820597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.820562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:05.835689 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.835658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qpnv\" (UniqueName: \"kubernetes.io/projected/67c4cb09-263d-4da4-913a-c6156fe0df60-kube-api-access-5qpnv\") pod \"67c4cb09-263d-4da4-913a-c6156fe0df60\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") "
Sep 16 14:34:05.835866 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.835721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c4cb09-263d-4da4-913a-c6156fe0df60-plugins-config-volume\") pod \"67c4cb09-263d-4da4-913a-c6156fe0df60\" (UID: \"67c4cb09-263d-4da4-913a-c6156fe0df60\") "
Sep 16 14:34:05.836211 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.836177    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67c4cb09-263d-4da4-913a-c6156fe0df60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67c4cb09-263d-4da4-913a-c6156fe0df60" (UID: "67c4cb09-263d-4da4-913a-c6156fe0df60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:05.838205 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.838178    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67c4cb09-263d-4da4-913a-c6156fe0df60-kube-api-access-5qpnv" (OuterVolumeSpecName: "kube-api-access-5qpnv") pod "67c4cb09-263d-4da4-913a-c6156fe0df60" (UID: "67c4cb09-263d-4da4-913a-c6156fe0df60"). InnerVolumeSpecName "kube-api-access-5qpnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:05.937245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.937207    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qpnv\" (UniqueName: \"kubernetes.io/projected/67c4cb09-263d-4da4-913a-c6156fe0df60-kube-api-access-5qpnv\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:05.937245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:05.937237    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c4cb09-263d-4da4-913a-c6156fe0df60-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:06.007946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.007911    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cfe4f51-d748-4bbb-8c87-b81728da1b27" path="/var/lib/kubelet/pods/3cfe4f51-d748-4bbb-8c87-b81728da1b27/volumes"
Sep 16 14:34:06.166931 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.166829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:06.265330 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.265282    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:06.434290 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.434198    2580 generic.go:356] "Generic (PLEG): container finished" podID="67c4cb09-263d-4da4-913a-c6156fe0df60" containerID="73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7" exitCode=0
Sep 16 14:34:06.434290 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.434250    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"
Sep 16 14:34:06.434496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.434293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm" event={"ID":"67c4cb09-263d-4da4-913a-c6156fe0df60","Type":"ContainerDied","Data":"73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7"}
Sep 16 14:34:06.434496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.434351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm" event={"ID":"67c4cb09-263d-4da4-913a-c6156fe0df60","Type":"ContainerDied","Data":"b93464ddfa4aabc8e58861b8eb48df9a7d0ea218ee7e92af5287ea2413df83f7"}
Sep 16 14:34:06.434496 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.434380    2580 scope.go:117] "RemoveContainer" containerID="73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7"
Sep 16 14:34:06.444178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.444127    2580 scope.go:117] "RemoveContainer" containerID="73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7"
Sep 16 14:34:06.444490 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:34:06.444468    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7\": container with ID starting with 73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7 not found: ID does not exist" containerID="73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7"
Sep 16 14:34:06.444569 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.444504    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7"} err="failed to get container status \"73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7\": rpc error: code = NotFound desc = could not find container \"73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7\": container with ID starting with 73b0482200f73b2f7f051a14bb1fad3845729df289b7f98ceb25777225c595c7 not found: ID does not exist"
Sep 16 14:34:06.453093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.453053    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"]
Sep 16 14:34:06.459517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:06.459483    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7b98f7-57598b6c58-tmbkm"]
Sep 16 14:34:07.021497 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:07.021456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:07.914178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:07.914120    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:08.006336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:08.006302    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67c4cb09-263d-4da4-913a-c6156fe0df60" path="/var/lib/kubelet/pods/67c4cb09-263d-4da4-913a-c6156fe0df60/volumes"
Sep 16 14:34:08.491381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:08.491334    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:34:08.491784 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:08.491413    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:34:08.503985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:08.503953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:34:08.835336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:08.835242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:09.464311 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:09.464283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:34:10.015063 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:10.015030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:10.868615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:10.868581    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:11.812886 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:11.812846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:12.958612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:12.958565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:13.068064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.068022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:13.271789 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.271702    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"]
Sep 16 14:34:13.271945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.271924    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" containerID="cri-o://b25cb4877046b5db84c34092254410fa39e7688d494d28570d7d3f0fedb04a42" gracePeriod=30
Sep 16 14:34:13.299591 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.299557    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:13.701169 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.701056    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"]
Sep 16 14:34:13.701606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.701592    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67c4cb09-263d-4da4-913a-c6156fe0df60" containerName="payload-pre-processing"
Sep 16 14:34:13.701651 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.701608    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67c4cb09-263d-4da4-913a-c6156fe0df60" containerName="payload-pre-processing"
Sep 16 14:34:13.701716 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.701705    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67c4cb09-263d-4da4-913a-c6156fe0df60" containerName="payload-pre-processing"
Sep 16 14:34:13.705558 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.705534    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"
Sep 16 14:34:13.709293 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.709272    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:34:13.709399 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.709332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:34:13.715708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.715681    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"]
Sep 16 14:34:13.823792 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.823751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/451f0f43-0151-4404-92d6-91701390f388-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-927f98-fp7wk\" (UID: \"451f0f43-0151-4404-92d6-91701390f388\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"
Sep 16 14:34:13.924639 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.924580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/451f0f43-0151-4404-92d6-91701390f388-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-927f98-fp7wk\" (UID: \"451f0f43-0151-4404-92d6-91701390f388\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"
Sep 16 14:34:13.925259 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:13.925238    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/451f0f43-0151-4404-92d6-91701390f388-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-927f98-fp7wk\" (UID: \"451f0f43-0151-4404-92d6-91701390f388\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"
Sep 16 14:34:14.017670 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:14.017634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"
Sep 16 14:34:14.061111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:14.061078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:14.297615 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:14.297584    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"]
Sep 16 14:34:14.298928 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:34:14.298902    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod451f0f43_0151_4404_92d6_91701390f388.slice/crio-495acd5cdfc7c1718e46b361ff0497ab4befb85719236b88af394bbc7890eca2 WatchSource:0}: Error finding container 495acd5cdfc7c1718e46b361ff0497ab4befb85719236b88af394bbc7890eca2: Status 404 returned error can't find the container with id 495acd5cdfc7c1718e46b361ff0497ab4befb85719236b88af394bbc7890eca2
Sep 16 14:34:14.485149 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:14.485101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk" event={"ID":"451f0f43-0151-4404-92d6-91701390f388","Type":"ContainerStarted","Data":"495acd5cdfc7c1718e46b361ff0497ab4befb85719236b88af394bbc7890eca2"}
Sep 16 14:34:15.118262 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:15.118225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:15.492751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:15.492716    2580 generic.go:356] "Generic (PLEG): container finished" podID="451f0f43-0151-4404-92d6-91701390f388" containerID="adaa68b9735c20004a9a77769dd7a3682fbf22d54fd6793675dd5d5bb85caa6d" exitCode=0
Sep 16 14:34:15.492933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:15.492799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk" event={"ID":"451f0f43-0151-4404-92d6-91701390f388","Type":"ContainerDied","Data":"adaa68b9735c20004a9a77769dd7a3682fbf22d54fd6793675dd5d5bb85caa6d"}
Sep 16 14:34:15.888314 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:15.888205    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:16.634991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:16.634960    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"
Sep 16 14:34:16.755365 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:16.755276    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/451f0f43-0151-4404-92d6-91701390f388-maas-api-service-ca\") pod \"451f0f43-0151-4404-92d6-91701390f388\" (UID: \"451f0f43-0151-4404-92d6-91701390f388\") "
Sep 16 14:34:16.755703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:16.755682    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/451f0f43-0151-4404-92d6-91701390f388-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "451f0f43-0151-4404-92d6-91701390f388" (UID: "451f0f43-0151-4404-92d6-91701390f388"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:16.808330 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:16.808284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:16.857039 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:16.857000    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/451f0f43-0151-4404-92d6-91701390f388-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:17.179450 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:17.179366    2580 patch_prober.go:28] interesting pod/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 14:34:17.179450 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:17.179426    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 14:34:17.505183 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:17.505148    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"
Sep 16 14:34:17.505338 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:17.505149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk" event={"ID":"451f0f43-0151-4404-92d6-91701390f388","Type":"ContainerDied","Data":"495acd5cdfc7c1718e46b361ff0497ab4befb85719236b88af394bbc7890eca2"}
Sep 16 14:34:17.505338 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:17.505279    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="495acd5cdfc7c1718e46b361ff0497ab4befb85719236b88af394bbc7890eca2"
Sep 16 14:34:17.616188 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:17.616121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:18.514737 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.514697    2580 generic.go:356] "Generic (PLEG): container finished" podID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerID="b25cb4877046b5db84c34092254410fa39e7688d494d28570d7d3f0fedb04a42" exitCode=0
Sep 16 14:34:18.515164 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.514764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" event={"ID":"07942cde-c97c-4436-bcd0-a6a4d054d7c1","Type":"ContainerDied","Data":"b25cb4877046b5db84c34092254410fa39e7688d494d28570d7d3f0fedb04a42"}
Sep 16 14:34:18.515164 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.514830    2580 scope.go:117] "RemoveContainer" containerID="45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168"
Sep 16 14:34:18.528840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.528810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:18.543303 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:34:18.543252    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168\": container with ID starting with 45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168 not found: ID does not exist" containerID="45b3c443de4e928b9ac648c5a1302b461ecf979868daf769d4d334b42bd86168"
Sep 16 14:34:18.546742 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.546723    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:34:18.674768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.674732    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-data\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.674768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.674775    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-token\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.674796    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-credential-socket\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.674824    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istiod-ca-cert\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.674845    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-socket\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675014 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.674973    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-envoy\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675037    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-certs\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675063    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:18.675260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:18.675260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675177    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr57h\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-kube-api-access-sr57h\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:18.675260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675232    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-podinfo\") pod \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\" (UID: \"07942cde-c97c-4436-bcd0-a6a4d054d7c1\") "
Sep 16 14:34:18.675533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675362    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:18.675533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:18.675729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675703    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-data\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.675729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675730    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-credential-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.675924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675744    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istiod-ca-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.675924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675757    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.675924 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.675769    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-workload-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.677355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.677329    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:18.677448 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.677345    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:18.677448 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.677354    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-kube-api-access-sr57h" (OuterVolumeSpecName: "kube-api-access-sr57h") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "kube-api-access-sr57h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:18.677448 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.677337    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07942cde-c97c-4436-bcd0-a6a4d054d7c1" (UID: "07942cde-c97c-4436-bcd0-a6a4d054d7c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:34:18.776896 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.776859    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-envoy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.776896 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.776889    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr57h\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-kube-api-access-sr57h\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.777103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.776899    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-podinfo\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:18.777103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:18.776926    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07942cde-c97c-4436-bcd0-a6a4d054d7c1-istio-token\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:19.478563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:19.478528    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:19.524461 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:19.524426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"
Sep 16 14:34:19.524933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:19.524462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds" event={"ID":"07942cde-c97c-4436-bcd0-a6a4d054d7c1","Type":"ContainerDied","Data":"e462a2b6eeb55dfafc56e5275b98a4666fff732b96b704dd7f4c47cda974edff"}
Sep 16 14:34:19.524933 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:19.524535    2580 scope.go:117] "RemoveContainer" containerID="b25cb4877046b5db84c34092254410fa39e7688d494d28570d7d3f0fedb04a42"
Sep 16 14:34:19.552600 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:19.552563    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"]
Sep 16 14:34:19.561345 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:19.561311    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7b98f7-openshift-default-78c65b8b8b-g75ds"]
Sep 16 14:34:20.006832 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:20.006796    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" path="/var/lib/kubelet/pods/07942cde-c97c-4436-bcd0-a6a4d054d7c1/volumes"
Sep 16 14:34:20.423485 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:20.423389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:23.352803 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:23.352710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:23.433411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:23.433372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:25.251890 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:25.251847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:29.092234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:29.092200    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:34:29.504952 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:29.504919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:29.689997 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:29.689963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:29.813040 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:29.812878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:29.816174 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:29.816111    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"]
Sep 16 14:34:29.818561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:29.818533    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-927f98-fp7wk"]
Sep 16 14:34:29.945344 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:29.945311    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:30.007718 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:30.007683    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="451f0f43-0151-4404-92d6-91701390f388" path="/var/lib/kubelet/pods/451f0f43-0151-4404-92d6-91701390f388/volumes"
Sep 16 14:34:30.238567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:30.238531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:30.392635 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:30.392597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:31.685272 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:31.685240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:32.646207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:32.646172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:33.790750 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:33.790713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:34.724830 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:34.724798    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:35.651478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:35.651441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:35.758735 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:35.758701    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh_c963f4c2-fab1-4b97-b519-3c7a12008de5/payload-processing/0.log"
Sep 16 14:34:35.874678 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:35.874645    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-28hd7_e27dd0dd-1f25-4d76-84de-32602bba121c/payload-processing/0.log"
Sep 16 14:34:36.015125 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:36.015090    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-28hd7_e27dd0dd-1f25-4d76-84de-32602bba121c/payload-processing/0.log"
Sep 16 14:34:36.136185 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:36.136134    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr_72125b19-c7ae-4f80-8d62-b0cdd93cab30/payload-processing/0.log"
Sep 16 14:34:36.264451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:36.264421    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr_72125b19-c7ae-4f80-8d62-b0cdd93cab30/payload-processing/0.log"
Sep 16 14:34:36.636665 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:36.636625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:36.987765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:36.987730    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:37.161625 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:37.161576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:37.306262 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:37.306169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:37.388400 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:37.388364    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:37.598382 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:37.598291    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"]
Sep 16 14:34:37.598578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:37.598508    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy" containerID="cri-o://b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44" gracePeriod=30
Sep 16 14:34:37.617587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:37.617553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:38.008040 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008008    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"]
Sep 16 14:34:38.008507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008492    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy"
Sep 16 14:34:38.008551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008508    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy"
Sep 16 14:34:38.008551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008517    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="451f0f43-0151-4404-92d6-91701390f388" containerName="revoke-keys"
Sep 16 14:34:38.008551 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008523    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="451f0f43-0151-4404-92d6-91701390f388" containerName="revoke-keys"
Sep 16 14:34:38.008649 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008599    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy"
Sep 16 14:34:38.008649 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008609    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy"
Sep 16 14:34:38.008649 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.008619    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="451f0f43-0151-4404-92d6-91701390f388" containerName="revoke-keys"
Sep 16 14:34:38.011872 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.011855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"
Sep 16 14:34:38.014799 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.014780    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:34:38.014936 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.014916    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:34:38.023312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.023292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"]
Sep 16 14:34:38.078402 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.078368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9474e123-9830-45f4-97a3-a87e07f02e6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22\" (UID: \"9474e123-9830-45f4-97a3-a87e07f02e6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"
Sep 16 14:34:38.178917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.178882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9474e123-9830-45f4-97a3-a87e07f02e6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22\" (UID: \"9474e123-9830-45f4-97a3-a87e07f02e6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"
Sep 16 14:34:38.179561 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.179543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9474e123-9830-45f4-97a3-a87e07f02e6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22\" (UID: \"9474e123-9830-45f4-97a3-a87e07f02e6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"
Sep 16 14:34:38.210387 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.210339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:38.323651 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.323564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"
Sep 16 14:34:38.608381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.608358    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"]
Sep 16 14:34:38.610351 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:34:38.610321    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9474e123_9830_45f4_97a3_a87e07f02e6a.slice/crio-dbc4a02f72442daf0aaf8814105c61a0d27757dd6fd5ff492989f397bc9f743e WatchSource:0}: Error finding container dbc4a02f72442daf0aaf8814105c61a0d27757dd6fd5ff492989f397bc9f743e: Status 404 returned error can't find the container with id dbc4a02f72442daf0aaf8814105c61a0d27757dd6fd5ff492989f397bc9f743e
Sep 16 14:34:38.638965 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:38.638937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22" event={"ID":"9474e123-9830-45f4-97a3-a87e07f02e6a","Type":"ContainerStarted","Data":"dbc4a02f72442daf0aaf8814105c61a0d27757dd6fd5ff492989f397bc9f743e"}
Sep 16 14:34:39.299616 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:39.299584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:39.646038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:39.645946    2580 generic.go:356] "Generic (PLEG): container finished" podID="9474e123-9830-45f4-97a3-a87e07f02e6a" containerID="10d67981c6b6fb2b49cfa5f4b942625501fddc1b7d8837691b825a9ac0f21d16" exitCode=0
Sep 16 14:34:39.646218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:39.646045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22" event={"ID":"9474e123-9830-45f4-97a3-a87e07f02e6a","Type":"ContainerDied","Data":"10d67981c6b6fb2b49cfa5f4b942625501fddc1b7d8837691b825a9ac0f21d16"}
Sep 16 14:34:40.403113 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:40.403066    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:40.510203 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:40.510132    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:40.606045 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:40.606005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:40.793225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:40.793202    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"
Sep 16 14:34:40.804244 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:40.804217    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9474e123-9830-45f4-97a3-a87e07f02e6a-maas-api-service-ca\") pod \"9474e123-9830-45f4-97a3-a87e07f02e6a\" (UID: \"9474e123-9830-45f4-97a3-a87e07f02e6a\") "
Sep 16 14:34:40.804533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:40.804511    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9474e123-9830-45f4-97a3-a87e07f02e6a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9474e123-9830-45f4-97a3-a87e07f02e6a" (UID: "9474e123-9830-45f4-97a3-a87e07f02e6a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:40.905878 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:40.905791    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9474e123-9830-45f4-97a3-a87e07f02e6a-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:41.649463 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:41.649430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:41.660271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:41.660241    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"
Sep 16 14:34:41.660271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:41.660259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22" event={"ID":"9474e123-9830-45f4-97a3-a87e07f02e6a","Type":"ContainerDied","Data":"dbc4a02f72442daf0aaf8814105c61a0d27757dd6fd5ff492989f397bc9f743e"}
Sep 16 14:34:41.660506 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:41.660313    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbc4a02f72442daf0aaf8814105c61a0d27757dd6fd5ff492989f397bc9f743e"
Sep 16 14:34:42.595594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.595563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:42.851319 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.851253    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:34:42.928541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-socket\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.928541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928546    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509b7b13-3e3a-45c3-856b-8bf53a9299df-istiod-ca-cert\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.928813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928585    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-credential-socket\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.928813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-data\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.928813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-token\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.928813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928722    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-podinfo\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.928813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-envoy\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.928813 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928777    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sspfk\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-kube-api-access-sspfk\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.929126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-certs\") pod \"509b7b13-3e3a-45c3-856b-8bf53a9299df\" (UID: \"509b7b13-3e3a-45c3-856b-8bf53a9299df\") "
Sep 16 14:34:42.929126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928842    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:42.929126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928979    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:42.929126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.928993    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/509b7b13-3e3a-45c3-856b-8bf53a9299df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:42.929407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.929249    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:42.929407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.929277    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-credential-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:42.929407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.929301    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:42.929407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.929316    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509b7b13-3e3a-45c3-856b-8bf53a9299df-istiod-ca-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:42.929407 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.929315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-data" (OuterVolumeSpecName: "istio-data") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:42.930975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.930953    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-kube-api-access-sspfk" (OuterVolumeSpecName: "kube-api-access-sspfk") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "kube-api-access-sspfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:42.930975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.930949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-token" (OuterVolumeSpecName: "istio-token") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:42.931405 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.931378    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:34:42.931514 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:42.931380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "509b7b13-3e3a-45c3-856b-8bf53a9299df" (UID: "509b7b13-3e3a-45c3-856b-8bf53a9299df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:34:43.030230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.030195    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-podinfo\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:43.030230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.030224    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-envoy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:43.030230 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.030233    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sspfk\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-kube-api-access-sspfk\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:43.030481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.030243    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-workload-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:43.030481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.030251    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-data\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:43.030481 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.030260    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509b7b13-3e3a-45c3-856b-8bf53a9299df-istio-token\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:43.521948 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.521898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:43.672989 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.672953    2580 generic.go:356] "Generic (PLEG): container finished" podID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerID="b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44" exitCode=0
Sep 16 14:34:43.673184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.673011    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"
Sep 16 14:34:43.673184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.673032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" event={"ID":"509b7b13-3e3a-45c3-856b-8bf53a9299df","Type":"ContainerDied","Data":"b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44"}
Sep 16 14:34:43.673184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.673089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd" event={"ID":"509b7b13-3e3a-45c3-856b-8bf53a9299df","Type":"ContainerDied","Data":"94c60300232677d3df23fb459dcac08a0b174cdf69538d41bdc48e1cd377b80e"}
Sep 16 14:34:43.673184 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.673114    2580 scope.go:117] "RemoveContainer" containerID="b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44"
Sep 16 14:34:43.683211 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.683188    2580 scope.go:117] "RemoveContainer" containerID="5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221"
Sep 16 14:34:43.694185 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.694162    2580 scope.go:117] "RemoveContainer" containerID="b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44"
Sep 16 14:34:43.694515 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:34:43.694491    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44\": container with ID starting with b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44 not found: ID does not exist" containerID="b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44"
Sep 16 14:34:43.694559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.694525    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44"} err="failed to get container status \"b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44\": rpc error: code = NotFound desc = could not find container \"b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44\": container with ID starting with b251a8216466c2f57832391fda427d10504eeb46025cdcb7475f671870f2df44 not found: ID does not exist"
Sep 16 14:34:43.694559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.694546    2580 scope.go:117] "RemoveContainer" containerID="5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221"
Sep 16 14:34:43.694831 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:34:43.694805    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221\": container with ID starting with 5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221 not found: ID does not exist" containerID="5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221"
Sep 16 14:34:43.694897 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.694830    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221"} err="failed to get container status \"5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221\": rpc error: code = NotFound desc = could not find container \"5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221\": container with ID starting with 5bf640cafaa3d8fb0d05317f134d58aa1505bc5599decab3d626ba50d8201221 not found: ID does not exist"
Sep 16 14:34:43.699687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.699660    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"]
Sep 16 14:34:43.702046 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:43.702026    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-927f98-openshift-default-fcb5dd748-gzvnd"]
Sep 16 14:34:44.014969 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:44.014933    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" path="/var/lib/kubelet/pods/509b7b13-3e3a-45c3-856b-8bf53a9299df/volumes"
Sep 16 14:34:44.364251 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:44.364122    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:45.473108 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:45.473067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:45.567521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:45.567484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:45.739671 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:45.739569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:46.151583 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.151476    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:46.251453 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.251401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:46.278801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.278768    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"]
Sep 16 14:34:46.279242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279230    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy"
Sep 16 14:34:46.279295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279244    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy"
Sep 16 14:34:46.279295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279264    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy"
Sep 16 14:34:46.279295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279270    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07942cde-c97c-4436-bcd0-a6a4d054d7c1" containerName="istio-proxy"
Sep 16 14:34:46.279295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279291    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9474e123-9830-45f4-97a3-a87e07f02e6a" containerName="revoke-keys"
Sep 16 14:34:46.279428 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279299    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9474e123-9830-45f4-97a3-a87e07f02e6a" containerName="revoke-keys"
Sep 16 14:34:46.279428 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279358    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy"
Sep 16 14:34:46.279428 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.279366    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9474e123-9830-45f4-97a3-a87e07f02e6a" containerName="revoke-keys"
Sep 16 14:34:46.284038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.284014    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.287439 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.287415    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"]
Sep 16 14:34:46.287959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.287938    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-80d46a\""
Sep 16 14:34:46.288064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.287958    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy"
Sep 16 14:34:46.288064 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.287973    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy"
Sep 16 14:34:46.288159 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.288086    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="509b7b13-3e3a-45c3-856b-8bf53a9299df" containerName="istio-proxy"
Sep 16 14:34:46.291274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.291250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.297335 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.297307    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"]
Sep 16 14:34:46.306312 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.306284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"]
Sep 16 14:34:46.362926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.362894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889bae51-46b0-45ff-952e-f6f1f18d5df8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.363099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.362942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8kh6\" (UniqueName: \"kubernetes.io/projected/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-kube-api-access-b8kh6\") pod \"payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.363099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.363043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.363099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.363089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnxj2\" (UniqueName: \"kubernetes.io/projected/889bae51-46b0-45ff-952e-f6f1f18d5df8-kube-api-access-pnxj2\") pod \"payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.464239 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.464202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnxj2\" (UniqueName: \"kubernetes.io/projected/889bae51-46b0-45ff-952e-f6f1f18d5df8-kube-api-access-pnxj2\") pod \"payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.464415 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.464355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889bae51-46b0-45ff-952e-f6f1f18d5df8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.464415 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.464398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8kh6\" (UniqueName: \"kubernetes.io/projected/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-kube-api-access-b8kh6\") pod \"payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.464840 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.464540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.465229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.465200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.465229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.465221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889bae51-46b0-45ff-952e-f6f1f18d5df8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.473808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.473782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8kh6\" (UniqueName: \"kubernetes.io/projected/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-kube-api-access-b8kh6\") pod \"payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.474118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.473785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnxj2\" (UniqueName: \"kubernetes.io/projected/889bae51-46b0-45ff-952e-f6f1f18d5df8-kube-api-access-pnxj2\") pod \"payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.595954 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.595918    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:46.605807 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.605774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:46.887694 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.887668    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"]
Sep 16 14:34:46.889378 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:34:46.889351    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fcc85e_7e49_44df_ad7f_8bacc1f3054f.slice/crio-ea99941b7e273e8cb7669b2a1f85a6160ec8b3e97c63c4ed3107260de6b500ac WatchSource:0}: Error finding container ea99941b7e273e8cb7669b2a1f85a6160ec8b3e97c63c4ed3107260de6b500ac: Status 404 returned error can't find the container with id ea99941b7e273e8cb7669b2a1f85a6160ec8b3e97c63c4ed3107260de6b500ac
Sep 16 14:34:46.895950 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.895931    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"]
Sep 16 14:34:46.896521 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:34:46.896492    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889bae51_46b0_45ff_952e_f6f1f18d5df8.slice/crio-6500a633968b49d8a6dd4ca5e68a4102d17f456d0768305a355f43fdc1350434 WatchSource:0}: Error finding container 6500a633968b49d8a6dd4ca5e68a4102d17f456d0768305a355f43fdc1350434: Status 404 returned error can't find the container with id 6500a633968b49d8a6dd4ca5e68a4102d17f456d0768305a355f43fdc1350434
Sep 16 14:34:46.898033 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:46.898012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:47.560889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.560839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:47.698356 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.698321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq" event={"ID":"889bae51-46b0-45ff-952e-f6f1f18d5df8","Type":"ContainerStarted","Data":"34b08f78b4165f94aa5956a68010f173b1244a7c12588a99998662667a055c9f"}
Sep 16 14:34:47.698549 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.698377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq" event={"ID":"889bae51-46b0-45ff-952e-f6f1f18d5df8","Type":"ContainerStarted","Data":"6500a633968b49d8a6dd4ca5e68a4102d17f456d0768305a355f43fdc1350434"}
Sep 16 14:34:47.698549 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.698514    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:47.700042 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.700020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz" event={"ID":"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f","Type":"ContainerStarted","Data":"b617f650ff04a7535426dc3529595a21bcdde88651e7a71b67b5b9b227db1cc6"}
Sep 16 14:34:47.700198 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.700067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz" event={"ID":"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f","Type":"ContainerStarted","Data":"ea99941b7e273e8cb7669b2a1f85a6160ec8b3e97c63c4ed3107260de6b500ac"}
Sep 16 14:34:47.700198 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.700097    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:47.733942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.733888    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz" podStartSLOduration=1.733870169 podStartE2EDuration="1.733870169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:34:47.732207491 +0000 UTC m=+1818.220217567" watchObservedRunningTime="2026-09-16 14:34:47.733870169 +0000 UTC m=+1818.221880284"
Sep 16 14:34:47.734765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:47.734735    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq" podStartSLOduration=1.734723558 podStartE2EDuration="1.734723558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:34:47.716192359 +0000 UTC m=+1818.204202436" watchObservedRunningTime="2026-09-16 14:34:47.734723558 +0000 UTC m=+1818.222733669"
Sep 16 14:34:48.404429 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.404391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:48.406255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.406231    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"]
Sep 16 14:34:48.410556 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.410533    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-2f9be3-29826150-7zkng"]
Sep 16 14:34:48.446889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.446850    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"]
Sep 16 14:34:48.447130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.447101    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl" podUID="c41555e7-54df-4c04-bd6a-975d5cfda52d" containerName="payload-processing" containerID="cri-o://67d724299be3d2fa6d36ecf4a69e67d78187ea585ca5aa48bb3ec6ef9bb6dd16" gracePeriod=30
Sep 16 14:34:48.509192 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.509133    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:48.610023 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.609984    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:48.682563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.682471    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"]
Sep 16 14:34:48.684717 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.684696    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-2f9be3-6hj22"]
Sep 16 14:34:48.689257 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.689223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:48.707788 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.707751    2580 generic.go:356] "Generic (PLEG): container finished" podID="c41555e7-54df-4c04-bd6a-975d5cfda52d" containerID="67d724299be3d2fa6d36ecf4a69e67d78187ea585ca5aa48bb3ec6ef9bb6dd16" exitCode=0
Sep 16 14:34:48.707979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.707824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl" event={"ID":"c41555e7-54df-4c04-bd6a-975d5cfda52d","Type":"ContainerDied","Data":"67d724299be3d2fa6d36ecf4a69e67d78187ea585ca5aa48bb3ec6ef9bb6dd16"}
Sep 16 14:34:48.707979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.707923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl" event={"ID":"c41555e7-54df-4c04-bd6a-975d5cfda52d","Type":"ContainerDied","Data":"02f32c15ef6b874e740f19b7f3b5b18906815b091b909f66b1c4a08fe01ba010"}
Sep 16 14:34:48.707979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.707948    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02f32c15ef6b874e740f19b7f3b5b18906815b091b909f66b1c4a08fe01ba010"
Sep 16 14:34:48.717830 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.717800    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:34:48.792383 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.792343    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v5b4\" (UniqueName: \"kubernetes.io/projected/c41555e7-54df-4c04-bd6a-975d5cfda52d-kube-api-access-9v5b4\") pod \"c41555e7-54df-4c04-bd6a-975d5cfda52d\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") "
Sep 16 14:34:48.792592 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.792412    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41555e7-54df-4c04-bd6a-975d5cfda52d-plugins-config-volume\") pod \"c41555e7-54df-4c04-bd6a-975d5cfda52d\" (UID: \"c41555e7-54df-4c04-bd6a-975d5cfda52d\") "
Sep 16 14:34:48.793093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.792933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c41555e7-54df-4c04-bd6a-975d5cfda52d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c41555e7-54df-4c04-bd6a-975d5cfda52d" (UID: "c41555e7-54df-4c04-bd6a-975d5cfda52d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:48.793093 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.793033    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41555e7-54df-4c04-bd6a-975d5cfda52d-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:48.795198 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.795132    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c41555e7-54df-4c04-bd6a-975d5cfda52d-kube-api-access-9v5b4" (OuterVolumeSpecName: "kube-api-access-9v5b4") pod "c41555e7-54df-4c04-bd6a-975d5cfda52d" (UID: "c41555e7-54df-4c04-bd6a-975d5cfda52d"). InnerVolumeSpecName "kube-api-access-9v5b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:48.893852 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:48.893815    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v5b4\" (UniqueName: \"kubernetes.io/projected/c41555e7-54df-4c04-bd6a-975d5cfda52d-kube-api-access-9v5b4\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:49.065672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:49.065632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:49.714103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:49.714067    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"
Sep 16 14:34:49.743084 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:49.743050    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"]
Sep 16 14:34:49.748405 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:49.748377    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-2f9be3-77574c689c-5w4jl"]
Sep 16 14:34:49.928577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:49.928540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:50.007079 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:50.006999    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9474e123-9830-45f4-97a3-a87e07f02e6a" path="/var/lib/kubelet/pods/9474e123-9830-45f4-97a3-a87e07f02e6a/volumes"
Sep 16 14:34:50.007325 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:50.007311    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0113f57-043a-4669-9b8d-86bba7831d8b" path="/var/lib/kubelet/pods/a0113f57-043a-4669-9b8d-86bba7831d8b/volumes"
Sep 16 14:34:50.007605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:50.007594    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c41555e7-54df-4c04-bd6a-975d5cfda52d" path="/var/lib/kubelet/pods/c41555e7-54df-4c04-bd6a-975d5cfda52d/volumes"
Sep 16 14:34:50.938942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:50.938908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:52.556629 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:52.556585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:52.730077 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:52.730045    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:34:53.612919 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:53.612884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:54.526001 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:54.525959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:55.608703 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:55.608660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:55.706564 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:55.706520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:55.875300 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:55.875210    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"]
Sep 16 14:34:55.875524 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:55.875490    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy" containerID="cri-o://0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6" gracePeriod=30
Sep 16 14:34:55.903910 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:55.903873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:56.282237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.282206    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"]
Sep 16 14:34:56.282678 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.282664    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c41555e7-54df-4c04-bd6a-975d5cfda52d" containerName="payload-processing"
Sep 16 14:34:56.282721 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.282680    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c41555e7-54df-4c04-bd6a-975d5cfda52d" containerName="payload-processing"
Sep 16 14:34:56.282769 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.282760    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c41555e7-54df-4c04-bd6a-975d5cfda52d" containerName="payload-processing"
Sep 16 14:34:56.287515 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.287497    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"
Sep 16 14:34:56.290738 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.290718    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:34:56.290937 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.290783    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:34:56.294794 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.294740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"]
Sep 16 14:34:56.370706 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.370670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbb075d-4abb-4f7b-9132-246bfc6a416f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc\" (UID: \"efbb075d-4abb-4f7b-9132-246bfc6a416f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"
Sep 16 14:34:56.472096 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.472062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbb075d-4abb-4f7b-9132-246bfc6a416f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc\" (UID: \"efbb075d-4abb-4f7b-9132-246bfc6a416f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"
Sep 16 14:34:56.472739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.472720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbb075d-4abb-4f7b-9132-246bfc6a416f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc\" (UID: \"efbb075d-4abb-4f7b-9132-246bfc6a416f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"
Sep 16 14:34:56.599396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.599309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"
Sep 16 14:34:56.643351 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.643313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:56.885754 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:56.885729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"]
Sep 16 14:34:56.888625 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:34:56.888594    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefbb075d_4abb_4f7b_9132_246bfc6a416f.slice/crio-0725124d500adc19ced9c36b9c3dab0c4d7c4063fee66630ce0bebc27a3f7c22 WatchSource:0}: Error finding container 0725124d500adc19ced9c36b9c3dab0c4d7c4063fee66630ce0bebc27a3f7c22: Status 404 returned error can't find the container with id 0725124d500adc19ced9c36b9c3dab0c4d7c4063fee66630ce0bebc27a3f7c22
Sep 16 14:34:57.761753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:57.761716    2580 generic.go:356] "Generic (PLEG): container finished" podID="efbb075d-4abb-4f7b-9132-246bfc6a416f" containerID="c79bcb7efeea4d0a766198e6ae217849e6e1a8489af021fdbf0549cefe2c2080" exitCode=0
Sep 16 14:34:57.762165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:57.761804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc" event={"ID":"efbb075d-4abb-4f7b-9132-246bfc6a416f","Type":"ContainerDied","Data":"c79bcb7efeea4d0a766198e6ae217849e6e1a8489af021fdbf0549cefe2c2080"}
Sep 16 14:34:57.762165 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:57.761849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc" event={"ID":"efbb075d-4abb-4f7b-9132-246bfc6a416f","Type":"ContainerStarted","Data":"0725124d500adc19ced9c36b9c3dab0c4d7c4063fee66630ce0bebc27a3f7c22"}
Sep 16 14:34:57.784227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:57.784196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:58.710507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:58.710477    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:34:58.710708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:58.710551    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:34:58.711432 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:58.711405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:58.912127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:58.912102    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"
Sep 16 14:34:58.999875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:58.999784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbb075d-4abb-4f7b-9132-246bfc6a416f-maas-api-service-ca\") pod \"efbb075d-4abb-4f7b-9132-246bfc6a416f\" (UID: \"efbb075d-4abb-4f7b-9132-246bfc6a416f\") "
Sep 16 14:34:59.000196 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:59.000174    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efbb075d-4abb-4f7b-9132-246bfc6a416f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efbb075d-4abb-4f7b-9132-246bfc6a416f" (UID: "efbb075d-4abb-4f7b-9132-246bfc6a416f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:59.101089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:59.101053    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbb075d-4abb-4f7b-9132-246bfc6a416f-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:34:59.644758 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:59.644719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:34:59.774887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:59.774855    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"
Sep 16 14:34:59.775082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:59.774894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc" event={"ID":"efbb075d-4abb-4f7b-9132-246bfc6a416f","Type":"ContainerDied","Data":"0725124d500adc19ced9c36b9c3dab0c4d7c4063fee66630ce0bebc27a3f7c22"}
Sep 16 14:34:59.775082 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:34:59.774949    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0725124d500adc19ced9c36b9c3dab0c4d7c4063fee66630ce0bebc27a3f7c22"
Sep 16 14:35:00.348750 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:00.348714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:01.142119 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.142046    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:35:01.224563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224525    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-credential-socket\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.224563 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224564    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-podinfo\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.224825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224595    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-socket\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.224825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224640    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istiod-ca-cert\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.224825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224660    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-certs\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.224825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224706    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-token\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.224825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224763    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-envoy\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.224825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224791    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4hrk\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-kube-api-access-s4hrk\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.225128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224880    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-data\") pod \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\" (UID: \"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b\") "
Sep 16 14:35:01.225128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.224883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:01.225128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:01.225128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:01.225128 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:01.225455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225280    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:01.225455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225403    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-data\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.225455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225420    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-credential-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.225455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225436    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.225455 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225449    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istiod-ca-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.225629 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.225462    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-workload-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.227017 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.226986    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:01.227174 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.227082    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:01.227353 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.227328    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-kube-api-access-s4hrk" (OuterVolumeSpecName: "kube-api-access-s4hrk") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "kube-api-access-s4hrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:01.227353 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.227346    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" (UID: "2f6014b7-e2dd-4f53-8f4e-af84cb3e894b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:35:01.325766 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.325727    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:01.326227 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.326204    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-podinfo\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.326284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.326229    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-token\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.326284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.326238    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-istio-envoy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.326284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.326246    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4hrk\" (UniqueName: \"kubernetes.io/projected/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b-kube-api-access-s4hrk\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:01.787804 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.787767    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerID="0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6" exitCode=0
Sep 16 14:35:01.788242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.787818    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"
Sep 16 14:35:01.788242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.787858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" event={"ID":"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b","Type":"ContainerDied","Data":"0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6"}
Sep 16 14:35:01.788242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.787916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk" event={"ID":"2f6014b7-e2dd-4f53-8f4e-af84cb3e894b","Type":"ContainerDied","Data":"1bd59afad407e206df26adfd6994b87526e9838d51d9a65e545f7cdda3d891c7"}
Sep 16 14:35:01.788242 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.787946    2580 scope.go:117] "RemoveContainer" containerID="0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6"
Sep 16 14:35:01.799318 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.799301    2580 scope.go:117] "RemoveContainer" containerID="31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d"
Sep 16 14:35:01.807999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.807977    2580 scope.go:117] "RemoveContainer" containerID="0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6"
Sep 16 14:35:01.808310 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:35:01.808290    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6\": container with ID starting with 0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6 not found: ID does not exist" containerID="0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6"
Sep 16 14:35:01.808371 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.808320    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6"} err="failed to get container status \"0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6\": rpc error: code = NotFound desc = could not find container \"0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6\": container with ID starting with 0279d56b6235452a0ae03bf7ee4cae22dfff05d704369eda9a76331136e424b6 not found: ID does not exist"
Sep 16 14:35:01.808371 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.808340    2580 scope.go:117] "RemoveContainer" containerID="31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d"
Sep 16 14:35:01.808608 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:35:01.808595    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d\": container with ID starting with 31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d not found: ID does not exist" containerID="31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d"
Sep 16 14:35:01.808663 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.808610    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d"} err="failed to get container status \"31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d\": rpc error: code = NotFound desc = could not find container \"31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d\": container with ID starting with 31f7dd36a6893ac7ba7e5e8583090f0f6c66ee4b9955634360ee9d6b4662ff3d not found: ID does not exist"
Sep 16 14:35:01.815083 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.815053    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"]
Sep 16 14:35:01.817284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:01.817259    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-2f9be3-openshift-default-59697969f8-dh2xk"]
Sep 16 14:35:02.007376 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:02.007343    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" path="/var/lib/kubelet/pods/2f6014b7-e2dd-4f53-8f4e-af84cb3e894b/volumes"
Sep 16 14:35:02.213036 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:02.212999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:03.000871 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:03.000834    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:35:03.215022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:03.214987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:04.450235 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:04.450197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:05.519778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:05.519719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:06.363033 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:06.362998    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:07.195208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:07.195168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:07.931898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:07.931858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:08.015195 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.015160    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"]
Sep 16 14:35:08.015456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.015428    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7" podUID="470719d5-e9d5-4815-8cce-344097d1b877" containerName="payload-pre-processing" containerID="cri-o://ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32" gracePeriod=30
Sep 16 14:35:08.058673 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.058623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:08.158118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.158087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:08.259618 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.259561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:08.281623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.281594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:35:08.309202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.303709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2tsg\" (UniqueName: \"kubernetes.io/projected/470719d5-e9d5-4815-8cce-344097d1b877-kube-api-access-h2tsg\") pod \"470719d5-e9d5-4815-8cce-344097d1b877\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") "
Sep 16 14:35:08.309202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.303949    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/470719d5-e9d5-4815-8cce-344097d1b877-plugins-config-volume\") pod \"470719d5-e9d5-4815-8cce-344097d1b877\" (UID: \"470719d5-e9d5-4815-8cce-344097d1b877\") "
Sep 16 14:35:08.309202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.305590    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/470719d5-e9d5-4815-8cce-344097d1b877-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "470719d5-e9d5-4815-8cce-344097d1b877" (UID: "470719d5-e9d5-4815-8cce-344097d1b877"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:08.309202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.306492    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/470719d5-e9d5-4815-8cce-344097d1b877-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:08.316546 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.316469    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/470719d5-e9d5-4815-8cce-344097d1b877-kube-api-access-h2tsg" (OuterVolumeSpecName: "kube-api-access-h2tsg") pod "470719d5-e9d5-4815-8cce-344097d1b877" (UID: "470719d5-e9d5-4815-8cce-344097d1b877"). InnerVolumeSpecName "kube-api-access-h2tsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:08.407255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.407205    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2tsg\" (UniqueName: \"kubernetes.io/projected/470719d5-e9d5-4815-8cce-344097d1b877-kube-api-access-h2tsg\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:08.503038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.503004    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"]
Sep 16 14:35:08.512753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.512721    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-cbeb11-wdzcc"]
Sep 16 14:35:08.550555 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.550514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:08.808394 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.808299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:08.831485 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.831449    2580 generic.go:356] "Generic (PLEG): container finished" podID="470719d5-e9d5-4815-8cce-344097d1b877" containerID="ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32" exitCode=0
Sep 16 14:35:08.831682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.831514    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"
Sep 16 14:35:08.831682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.831532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7" event={"ID":"470719d5-e9d5-4815-8cce-344097d1b877","Type":"ContainerDied","Data":"ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32"}
Sep 16 14:35:08.831682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.831611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7" event={"ID":"470719d5-e9d5-4815-8cce-344097d1b877","Type":"ContainerDied","Data":"35cac17ec538a9bd6bd5e0adaf79a791e07c8e8ef91857f2188b673ee186fa50"}
Sep 16 14:35:08.831682 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.831634    2580 scope.go:117] "RemoveContainer" containerID="ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32"
Sep 16 14:35:08.841305 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.841286    2580 scope.go:117] "RemoveContainer" containerID="ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32"
Sep 16 14:35:08.841597 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:35:08.841577    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32\": container with ID starting with ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32 not found: ID does not exist" containerID="ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32"
Sep 16 14:35:08.841644 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.841607    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32"} err="failed to get container status \"ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32\": rpc error: code = NotFound desc = could not find container \"ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32\": container with ID starting with ad6d91ed10571202ec9387e091daa77e31fa05e4a23cd86054306beb3eff9a32 not found: ID does not exist"
Sep 16 14:35:08.855938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.855909    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"]
Sep 16 14:35:08.861786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:08.861758    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cbeb11-8688bd46c4-7tkp7"]
Sep 16 14:35:09.703951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:09.703903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:10.008086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:10.008051    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="470719d5-e9d5-4815-8cce-344097d1b877" path="/var/lib/kubelet/pods/470719d5-e9d5-4815-8cce-344097d1b877/volumes"
Sep 16 14:35:10.008443 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:10.008430    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efbb075d-4abb-4f7b-9132-246bfc6a416f" path="/var/lib/kubelet/pods/efbb075d-4abb-4f7b-9132-246bfc6a416f/volumes"
Sep 16 14:35:10.344121 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:10.344031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:10.574070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:10.574037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:11.664838 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:11.664802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:12.696559 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:12.696527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:13.707041 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:13.707003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:14.173581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:14.173537    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:35:20.007193 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:20.007154    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:35:30.390850 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:30.390819    2580 scope.go:117] "RemoveContainer" containerID="67d724299be3d2fa6d36ecf4a69e67d78187ea585ca5aa48bb3ec6ef9bb6dd16"
Sep 16 14:35:31.006127 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006087    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"]
Sep 16 14:35:31.006573 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006560    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470719d5-e9d5-4815-8cce-344097d1b877" containerName="payload-pre-processing"
Sep 16 14:35:31.006619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006575    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="470719d5-e9d5-4815-8cce-344097d1b877" containerName="payload-pre-processing"
Sep 16 14:35:31.006619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006587    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbb075d-4abb-4f7b-9132-246bfc6a416f" containerName="revoke-keys"
Sep 16 14:35:31.006619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006592    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbb075d-4abb-4f7b-9132-246bfc6a416f" containerName="revoke-keys"
Sep 16 14:35:31.006619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006603    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy"
Sep 16 14:35:31.006619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006609    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy"
Sep 16 14:35:31.006778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006624    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy"
Sep 16 14:35:31.006778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006629    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy"
Sep 16 14:35:31.006778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006688    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy"
Sep 16 14:35:31.006778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006697    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="470719d5-e9d5-4815-8cce-344097d1b877" containerName="payload-pre-processing"
Sep 16 14:35:31.006778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.006704    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="efbb075d-4abb-4f7b-9132-246bfc6a416f" containerName="revoke-keys"
Sep 16 14:35:31.011480 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.011457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"
Sep 16 14:35:31.015108 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.015082    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:35:31.015476 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.014794    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:35:31.025900 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.025862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"]
Sep 16 14:35:31.126877 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.126839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt\" (UID: \"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"
Sep 16 14:35:31.228161 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.228092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt\" (UID: \"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"
Sep 16 14:35:31.228727 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.228707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt\" (UID: \"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"
Sep 16 14:35:31.328875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.328774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"
Sep 16 14:35:31.602571 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.602543    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"]
Sep 16 14:35:31.605404 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:35:31.605371    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda54bafeb_6d23_4ef3_bb7a_e9f6f4bbb1b9.slice/crio-f4129f293d37be33aa6e63a8a0e67fc560e0967eea48283bf9b365ab6b1cc3fc WatchSource:0}: Error finding container f4129f293d37be33aa6e63a8a0e67fc560e0967eea48283bf9b365ab6b1cc3fc: Status 404 returned error can't find the container with id f4129f293d37be33aa6e63a8a0e67fc560e0967eea48283bf9b365ab6b1cc3fc
Sep 16 14:35:31.811720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.811687    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:31.968751 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.968712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt" event={"ID":"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9","Type":"ContainerStarted","Data":"573ed4d1fbcb733d0d1834e9f5772370e72bc7297c190e5dce7fa06c360ace52"}
Sep 16 14:35:31.968986 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.968773    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt" event={"ID":"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9","Type":"ContainerStarted","Data":"f4129f293d37be33aa6e63a8a0e67fc560e0967eea48283bf9b365ab6b1cc3fc"}
Sep 16 14:35:31.995653 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:31.995600    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt" podStartSLOduration=1.995582645 podStartE2EDuration="1.995582645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:35:31.99424724 +0000 UTC m=+1862.482257317" watchObservedRunningTime="2026-09-16 14:35:31.995582645 +0000 UTC m=+1862.483592788"
Sep 16 14:35:32.975575 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:32.975538    2580 generic.go:356] "Generic (PLEG): container finished" podID="a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9" containerID="573ed4d1fbcb733d0d1834e9f5772370e72bc7297c190e5dce7fa06c360ace52" exitCode=0
Sep 16 14:35:32.975942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:32.975624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt" event={"ID":"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9","Type":"ContainerDied","Data":"573ed4d1fbcb733d0d1834e9f5772370e72bc7297c190e5dce7fa06c360ace52"}
Sep 16 14:35:33.451283 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:33.451195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:33.677676 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:33.677632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:33.875711 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:33.875678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:34.126977 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.126900    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"
Sep 16 14:35:34.259638 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.259597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9-maas-api-service-ca\") pod \"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9\" (UID: \"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9\") "
Sep 16 14:35:34.259988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.259961    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9" (UID: "a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:34.360453 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.360417    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:34.628278 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.628243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:34.990543 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.990512    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"
Sep 16 14:35:34.990729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.990538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt" event={"ID":"a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9","Type":"ContainerDied","Data":"f4129f293d37be33aa6e63a8a0e67fc560e0967eea48283bf9b365ab6b1cc3fc"}
Sep 16 14:35:34.990729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:34.990601    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4129f293d37be33aa6e63a8a0e67fc560e0967eea48283bf9b365ab6b1cc3fc"
Sep 16 14:35:35.394476 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:35.394383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:40.674964 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:40.674916    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:40.775917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:40.775875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:41.084926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.084889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:41.770484 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.770449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:41.809731 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.809695    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"]
Sep 16 14:35:41.810035 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.809991    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq" podUID="889bae51-46b0-45ff-952e-f6f1f18d5df8" containerName="payload-processing" containerID="cri-o://34b08f78b4165f94aa5956a68010f173b1244a7c12588a99998662667a055c9f" gracePeriod=30
Sep 16 14:35:41.815475 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.815450    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"]
Sep 16 14:35:41.815868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.815839    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz" podUID="e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" containerName="payload-pre-processing" containerID="cri-o://b617f650ff04a7535426dc3529595a21bcdde88651e7a71b67b5b9b227db1cc6" gracePeriod=30
Sep 16 14:35:41.872987 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.872941    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:41.958825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:41.958733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:42.041914 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.041765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:42.042786 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.042726    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"]
Sep 16 14:35:42.045684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.045660    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-80d46a-h68dt"]
Sep 16 14:35:42.048597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.048569    2580 generic.go:356] "Generic (PLEG): container finished" podID="e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" containerID="b617f650ff04a7535426dc3529595a21bcdde88651e7a71b67b5b9b227db1cc6" exitCode=0
Sep 16 14:35:42.048698 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.048602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz" event={"ID":"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f","Type":"ContainerDied","Data":"b617f650ff04a7535426dc3529595a21bcdde88651e7a71b67b5b9b227db1cc6"}
Sep 16 14:35:42.050922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.050896    2580 generic.go:356] "Generic (PLEG): container finished" podID="889bae51-46b0-45ff-952e-f6f1f18d5df8" containerID="34b08f78b4165f94aa5956a68010f173b1244a7c12588a99998662667a055c9f" exitCode=0
Sep 16 14:35:42.051038 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.050998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq" event={"ID":"889bae51-46b0-45ff-952e-f6f1f18d5df8","Type":"ContainerDied","Data":"34b08f78b4165f94aa5956a68010f173b1244a7c12588a99998662667a055c9f"}
Sep 16 14:35:42.103527 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.103498    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:35:42.112280 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.112233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:42.131215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.131188    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:35:42.242488 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.242456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-plugins-config-volume\") pod \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") "
Sep 16 14:35:42.242488 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.242490    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnxj2\" (UniqueName: \"kubernetes.io/projected/889bae51-46b0-45ff-952e-f6f1f18d5df8-kube-api-access-pnxj2\") pod \"889bae51-46b0-45ff-952e-f6f1f18d5df8\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") "
Sep 16 14:35:42.242727 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.242537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889bae51-46b0-45ff-952e-f6f1f18d5df8-plugins-config-volume\") pod \"889bae51-46b0-45ff-952e-f6f1f18d5df8\" (UID: \"889bae51-46b0-45ff-952e-f6f1f18d5df8\") "
Sep 16 14:35:42.242727 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.242602    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8kh6\" (UniqueName: \"kubernetes.io/projected/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-kube-api-access-b8kh6\") pod \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\" (UID: \"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f\") "
Sep 16 14:35:42.242930 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.242902    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" (UID: "e8fcc85e-7e49-44df-ad7f-8bacc1f3054f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:42.243027 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.242906    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/889bae51-46b0-45ff-952e-f6f1f18d5df8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "889bae51-46b0-45ff-952e-f6f1f18d5df8" (UID: "889bae51-46b0-45ff-952e-f6f1f18d5df8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:42.244752 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.244726    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-kube-api-access-b8kh6" (OuterVolumeSpecName: "kube-api-access-b8kh6") pod "e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" (UID: "e8fcc85e-7e49-44df-ad7f-8bacc1f3054f"). InnerVolumeSpecName "kube-api-access-b8kh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:42.244844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.244822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/889bae51-46b0-45ff-952e-f6f1f18d5df8-kube-api-access-pnxj2" (OuterVolumeSpecName: "kube-api-access-pnxj2") pod "889bae51-46b0-45ff-952e-f6f1f18d5df8" (UID: "889bae51-46b0-45ff-952e-f6f1f18d5df8"). InnerVolumeSpecName "kube-api-access-pnxj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:42.344167 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.344072    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.344167 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.344100    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnxj2\" (UniqueName: \"kubernetes.io/projected/889bae51-46b0-45ff-952e-f6f1f18d5df8-kube-api-access-pnxj2\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.344167 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.344109    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889bae51-46b0-45ff-952e-f6f1f18d5df8-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.344167 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.344118    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8kh6\" (UniqueName: \"kubernetes.io/projected/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f-kube-api-access-b8kh6\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.405869 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.405827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:42.508296 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:42.508260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:43.058219 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.058185    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"
Sep 16 14:35:43.058656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.058178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz" event={"ID":"e8fcc85e-7e49-44df-ad7f-8bacc1f3054f","Type":"ContainerDied","Data":"ea99941b7e273e8cb7669b2a1f85a6160ec8b3e97c63c4ed3107260de6b500ac"}
Sep 16 14:35:43.058656 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.058345    2580 scope.go:117] "RemoveContainer" containerID="b617f650ff04a7535426dc3529595a21bcdde88651e7a71b67b5b9b227db1cc6"
Sep 16 14:35:43.059791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.059730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq" event={"ID":"889bae51-46b0-45ff-952e-f6f1f18d5df8","Type":"ContainerDied","Data":"6500a633968b49d8a6dd4ca5e68a4102d17f456d0768305a355f43fdc1350434"}
Sep 16 14:35:43.059791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.059737    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"
Sep 16 14:35:43.069265 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.069248    2580 scope.go:117] "RemoveContainer" containerID="34b08f78b4165f94aa5956a68010f173b1244a7c12588a99998662667a055c9f"
Sep 16 14:35:43.084765 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.084737    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"]
Sep 16 14:35:43.086896 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.086873    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-80d46a-5655cd84b9-64wxq"]
Sep 16 14:35:43.099734 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.099707    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"]
Sep 16 14:35:43.104264 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.104238    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-80d46a-8668fd8df8-cdfkz"]
Sep 16 14:35:43.526988 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:43.526938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:44.006764 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:44.006731    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="889bae51-46b0-45ff-952e-f6f1f18d5df8" path="/var/lib/kubelet/pods/889bae51-46b0-45ff-952e-f6f1f18d5df8/volumes"
Sep 16 14:35:44.007081 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:44.007069    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9" path="/var/lib/kubelet/pods/a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9/volumes"
Sep 16 14:35:44.007423 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:44.007411    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" path="/var/lib/kubelet/pods/e8fcc85e-7e49-44df-ad7f-8bacc1f3054f/volumes"
Sep 16 14:35:44.417877 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:44.417769    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:45.242952 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:45.242908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:45.573544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:45.573459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:35:48.632450 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:48.632413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:48.717297 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:48.717261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:48.906044 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:48.905956    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:52.564568 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.564486    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"]
Sep 16 14:35:52.565000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.564946    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9" containerName="revoke-keys"
Sep 16 14:35:52.565000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.564958    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9" containerName="revoke-keys"
Sep 16 14:35:52.565000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.564969    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="889bae51-46b0-45ff-952e-f6f1f18d5df8" containerName="payload-processing"
Sep 16 14:35:52.565000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.564975    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="889bae51-46b0-45ff-952e-f6f1f18d5df8" containerName="payload-processing"
Sep 16 14:35:52.565000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.564982    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" containerName="payload-pre-processing"
Sep 16 14:35:52.565000 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.564987    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" containerName="payload-pre-processing"
Sep 16 14:35:52.565237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.565057    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f6014b7-e2dd-4f53-8f4e-af84cb3e894b" containerName="istio-proxy"
Sep 16 14:35:52.565237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.565067    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="889bae51-46b0-45ff-952e-f6f1f18d5df8" containerName="payload-processing"
Sep 16 14:35:52.565237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.565078    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8fcc85e-7e49-44df-ad7f-8bacc1f3054f" containerName="payload-pre-processing"
Sep 16 14:35:52.565237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.565085    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a54bafeb-6d23-4ef3-bb7a-e9f6f4bbb1b9" containerName="revoke-keys"
Sep 16 14:35:52.569917 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.569900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"
Sep 16 14:35:52.572725 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.572698    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:35:52.572922 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.572906    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:35:52.578708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.578683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"]
Sep 16 14:35:52.643582 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.643543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d01a563-84e9-4c4b-bd5a-5a1b120929ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62\" (UID: \"2d01a563-84e9-4c4b-bd5a-5a1b120929ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"
Sep 16 14:35:52.744449 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.744408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d01a563-84e9-4c4b-bd5a-5a1b120929ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62\" (UID: \"2d01a563-84e9-4c4b-bd5a-5a1b120929ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"
Sep 16 14:35:52.745043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.745021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d01a563-84e9-4c4b-bd5a-5a1b120929ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62\" (UID: \"2d01a563-84e9-4c4b-bd5a-5a1b120929ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"
Sep 16 14:35:52.881972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:52.881876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"
Sep 16 14:35:53.158834 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:53.158807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"]
Sep 16 14:35:53.160885 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:35:53.160861    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d01a563_84e9_4c4b_bd5a_5a1b120929ca.slice/crio-fada09e93a11d1ee3636e66cdb42d215bf24ffeccdc6e51589c31520c95fbeae WatchSource:0}: Error finding container fada09e93a11d1ee3636e66cdb42d215bf24ffeccdc6e51589c31520c95fbeae: Status 404 returned error can't find the container with id fada09e93a11d1ee3636e66cdb42d215bf24ffeccdc6e51589c31520c95fbeae
Sep 16 14:35:53.745640 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:53.745541    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:54.131776 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:54.131681    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d01a563-84e9-4c4b-bd5a-5a1b120929ca" containerID="2c639fde32d90a573ea1216ee5721c69df698e2da950862fc66f07a4ba082c75" exitCode=0
Sep 16 14:35:54.131957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:54.131769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62" event={"ID":"2d01a563-84e9-4c4b-bd5a-5a1b120929ca","Type":"ContainerDied","Data":"2c639fde32d90a573ea1216ee5721c69df698e2da950862fc66f07a4ba082c75"}
Sep 16 14:35:54.131957 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:54.131824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62" event={"ID":"2d01a563-84e9-4c4b-bd5a-5a1b120929ca","Type":"ContainerStarted","Data":"fada09e93a11d1ee3636e66cdb42d215bf24ffeccdc6e51589c31520c95fbeae"}
Sep 16 14:35:54.558601 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:54.558563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:55.277207 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:55.277176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"
Sep 16 14:35:55.368478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:55.368448    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d01a563-84e9-4c4b-bd5a-5a1b120929ca-maas-api-service-ca\") pod \"2d01a563-84e9-4c4b-bd5a-5a1b120929ca\" (UID: \"2d01a563-84e9-4c4b-bd5a-5a1b120929ca\") "
Sep 16 14:35:55.368796 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:55.368772    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d01a563-84e9-4c4b-bd5a-5a1b120929ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d01a563-84e9-4c4b-bd5a-5a1b120929ca" (UID: "2d01a563-84e9-4c4b-bd5a-5a1b120929ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:55.470009 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:55.469968    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d01a563-84e9-4c4b-bd5a-5a1b120929ca-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:35:55.587875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:55.587829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:35:56.144292 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:56.144258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62" event={"ID":"2d01a563-84e9-4c4b-bd5a-5a1b120929ca","Type":"ContainerDied","Data":"fada09e93a11d1ee3636e66cdb42d215bf24ffeccdc6e51589c31520c95fbeae"}
Sep 16 14:35:56.144465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:56.144314    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fada09e93a11d1ee3636e66cdb42d215bf24ffeccdc6e51589c31520c95fbeae"
Sep 16 14:35:56.144465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:56.144267    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"
Sep 16 14:35:56.424839 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:35:56.424741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:04.842584 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:04.842552    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:04.992465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:04.992428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:05.883349 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:05.883312    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:36:08.282757 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.282725    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:08.314423 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.314390    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"]
Sep 16 14:36:08.314675 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.314643    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh" podUID="c963f4c2-fab1-4b97-b519-3c7a12008de5" containerName="payload-processing" containerID="cri-o://3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69" gracePeriod=30
Sep 16 14:36:08.382914 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.382879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:08.484375 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.484334    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:08.556396 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.556352    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"]
Sep 16 14:36:08.558460 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.558436    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b77fde-65q62"]
Sep 16 14:36:08.572412 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.572386    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:36:08.583659 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.583634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:08.698409 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.698369    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grg6n\" (UniqueName: \"kubernetes.io/projected/c963f4c2-fab1-4b97-b519-3c7a12008de5-kube-api-access-grg6n\") pod \"c963f4c2-fab1-4b97-b519-3c7a12008de5\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") "
Sep 16 14:36:08.698614 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.698460    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c963f4c2-fab1-4b97-b519-3c7a12008de5-plugins-config-volume\") pod \"c963f4c2-fab1-4b97-b519-3c7a12008de5\" (UID: \"c963f4c2-fab1-4b97-b519-3c7a12008de5\") "
Sep 16 14:36:08.698874 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.698848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c963f4c2-fab1-4b97-b519-3c7a12008de5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c963f4c2-fab1-4b97-b519-3c7a12008de5" (UID: "c963f4c2-fab1-4b97-b519-3c7a12008de5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:36:08.700576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.700550    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c963f4c2-fab1-4b97-b519-3c7a12008de5-kube-api-access-grg6n" (OuterVolumeSpecName: "kube-api-access-grg6n") pod "c963f4c2-fab1-4b97-b519-3c7a12008de5" (UID: "c963f4c2-fab1-4b97-b519-3c7a12008de5"). InnerVolumeSpecName "kube-api-access-grg6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:36:08.799825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.799730    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grg6n\" (UniqueName: \"kubernetes.io/projected/c963f4c2-fab1-4b97-b519-3c7a12008de5-kube-api-access-grg6n\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:08.799825 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.799764    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c963f4c2-fab1-4b97-b519-3c7a12008de5-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:08.936174 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:08.936118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:09.032963 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.032922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:09.216348 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.216312    2580 generic.go:356] "Generic (PLEG): container finished" podID="c963f4c2-fab1-4b97-b519-3c7a12008de5" containerID="3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69" exitCode=0
Sep 16 14:36:09.216508 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.216372    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"
Sep 16 14:36:09.216508 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.216372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh" event={"ID":"c963f4c2-fab1-4b97-b519-3c7a12008de5","Type":"ContainerDied","Data":"3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69"}
Sep 16 14:36:09.216508 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.216491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh" event={"ID":"c963f4c2-fab1-4b97-b519-3c7a12008de5","Type":"ContainerDied","Data":"6be1c275e1c2ba9c29b2737aab24e014e36409e9739d924079a70c74ff451444"}
Sep 16 14:36:09.216664 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.216512    2580 scope.go:117] "RemoveContainer" containerID="3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69"
Sep 16 14:36:09.226368 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.226349    2580 scope.go:117] "RemoveContainer" containerID="3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69"
Sep 16 14:36:09.226669 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:36:09.226649    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69\": container with ID starting with 3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69 not found: ID does not exist" containerID="3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69"
Sep 16 14:36:09.226726 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.226677    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69"} err="failed to get container status \"3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69\": rpc error: code = NotFound desc = could not find container \"3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69\": container with ID starting with 3d460fec7525823af2f76ea9815f5c80315a8c203e15a9b2e0369e49f1c81c69 not found: ID does not exist"
Sep 16 14:36:09.270176 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.270126    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"]
Sep 16 14:36:09.277592 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.277562    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b77fde-8588fc9bdc-k7njh"]
Sep 16 14:36:09.943375 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:09.943339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:10.007787 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:10.007753    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d01a563-84e9-4c4b-bd5a-5a1b120929ca" path="/var/lib/kubelet/pods/2d01a563-84e9-4c4b-bd5a-5a1b120929ca/volumes"
Sep 16 14:36:10.008252 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:10.008237    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c963f4c2-fab1-4b97-b519-3c7a12008de5" path="/var/lib/kubelet/pods/c963f4c2-fab1-4b97-b519-3c7a12008de5/volumes"
Sep 16 14:36:10.734991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:10.734940    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:14.034276 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:14.034247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"]
Sep 16 14:36:14.122709 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:14.122675    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:19.971679 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:19.971646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:36:23.000430 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:23.000393    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:36:29.053606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.053560    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp" podUID="acca96f1-fa3e-4350-92fa-aa3f074e75e0" containerName="main" containerID="cri-o://2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138" gracePeriod=45
Sep 16 14:36:29.312072 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.312002    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:36:29.331130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.331063    2580 generic.go:356] "Generic (PLEG): container finished" podID="acca96f1-fa3e-4350-92fa-aa3f074e75e0" containerID="2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138" exitCode=0
Sep 16 14:36:29.331130 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.331124    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"
Sep 16 14:36:29.331393 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.331126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp" event={"ID":"acca96f1-fa3e-4350-92fa-aa3f074e75e0","Type":"ContainerDied","Data":"2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138"}
Sep 16 14:36:29.331393 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.331215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp" event={"ID":"acca96f1-fa3e-4350-92fa-aa3f074e75e0","Type":"ContainerDied","Data":"80a6e295fbb2bfacb87bbb8ae8b965206a8e841d9b1407419143c0d6c1e9d19c"}
Sep 16 14:36:29.331393 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.331264    2580 scope.go:117] "RemoveContainer" containerID="2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138"
Sep 16 14:36:29.345636 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.345610    2580 scope.go:117] "RemoveContainer" containerID="2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138"
Sep 16 14:36:29.346133 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:36:29.346110    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138\": container with ID starting with 2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138 not found: ID does not exist" containerID="2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138"
Sep 16 14:36:29.346133 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.346177    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138"} err="failed to get container status \"2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138\": rpc error: code = NotFound desc = could not find container \"2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138\": container with ID starting with 2cc0115ff1b6da0d4f3306fbfb8a38935b4c101f6d8eff061200b65f9e0df138 not found: ID does not exist"
Sep 16 14:36:29.408307 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408277    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcbh7\" (UniqueName: \"kubernetes.io/projected/acca96f1-fa3e-4350-92fa-aa3f074e75e0-kube-api-access-tcbh7\") pod \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") "
Sep 16 14:36:29.408507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408346    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tls-certs\") pod \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") "
Sep 16 14:36:29.408507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408372    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-model-cache\") pod \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") "
Sep 16 14:36:29.408507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408413    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tmp-dir\") pod \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") "
Sep 16 14:36:29.408507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408475    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-dshm\") pod \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") "
Sep 16 14:36:29.408507 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408502    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-home\") pod \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\" (UID: \"acca96f1-fa3e-4350-92fa-aa3f074e75e0\") "
Sep 16 14:36:29.408778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408679    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "acca96f1-fa3e-4350-92fa-aa3f074e75e0" (UID: "acca96f1-fa3e-4350-92fa-aa3f074e75e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:29.408778 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408729    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "acca96f1-fa3e-4350-92fa-aa3f074e75e0" (UID: "acca96f1-fa3e-4350-92fa-aa3f074e75e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:29.408892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-home" (OuterVolumeSpecName: "home") pod "acca96f1-fa3e-4350-92fa-aa3f074e75e0" (UID: "acca96f1-fa3e-4350-92fa-aa3f074e75e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:29.408892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408842    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:29.408892 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.408858    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:29.410681 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.410653    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "acca96f1-fa3e-4350-92fa-aa3f074e75e0" (UID: "acca96f1-fa3e-4350-92fa-aa3f074e75e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:36:29.410681 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.410672    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acca96f1-fa3e-4350-92fa-aa3f074e75e0-kube-api-access-tcbh7" (OuterVolumeSpecName: "kube-api-access-tcbh7") pod "acca96f1-fa3e-4350-92fa-aa3f074e75e0" (UID: "acca96f1-fa3e-4350-92fa-aa3f074e75e0"). InnerVolumeSpecName "kube-api-access-tcbh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:36:29.410863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.410658    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-dshm" (OuterVolumeSpecName: "dshm") pod "acca96f1-fa3e-4350-92fa-aa3f074e75e0" (UID: "acca96f1-fa3e-4350-92fa-aa3f074e75e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:29.509445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.509404    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:29.509445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.509438    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acca96f1-fa3e-4350-92fa-aa3f074e75e0-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:29.509445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.509448    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcbh7\" (UniqueName: \"kubernetes.io/projected/acca96f1-fa3e-4350-92fa-aa3f074e75e0-kube-api-access-tcbh7\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:29.509690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.509458    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acca96f1-fa3e-4350-92fa-aa3f074e75e0-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:36:29.655550 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.655513    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"]
Sep 16 14:36:29.659221 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:29.659194    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-b77fde/ipp-route-b77fde-kserve-6b5f8997c5-j7spp"]
Sep 16 14:36:30.008063 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:30.008022    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acca96f1-fa3e-4350-92fa-aa3f074e75e0" path="/var/lib/kubelet/pods/acca96f1-fa3e-4350-92fa-aa3f074e75e0/volumes"
Sep 16 14:36:30.425833 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:30.425748    2580 scope.go:117] "RemoveContainer" containerID="045e107e3ed19ccc36c7a417caa97fb8e62d9a8d9c081f705c7751c9a029f215"
Sep 16 14:36:30.434351 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:30.434335    2580 scope.go:117] "RemoveContainer" containerID="5ca56c1502fb2c60b404e80d03bcc07708601754251e19831c2c4593ae2f9552"
Sep 16 14:36:43.000792 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:43.000757    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:36:51.011213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:51.011172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:52.341285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:52.341231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:58.671008 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:58.670936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:58.794544 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:58.794495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:58.996517 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:58.996486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:36:59.379215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379109    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"]
Sep 16 14:36:59.379676 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379656    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d01a563-84e9-4c4b-bd5a-5a1b120929ca" containerName="revoke-keys"
Sep 16 14:36:59.379772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379679    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d01a563-84e9-4c4b-bd5a-5a1b120929ca" containerName="revoke-keys"
Sep 16 14:36:59.379772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379698    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acca96f1-fa3e-4350-92fa-aa3f074e75e0" containerName="main"
Sep 16 14:36:59.379772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379706    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="acca96f1-fa3e-4350-92fa-aa3f074e75e0" containerName="main"
Sep 16 14:36:59.379772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379743    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c963f4c2-fab1-4b97-b519-3c7a12008de5" containerName="payload-processing"
Sep 16 14:36:59.379772 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379754    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c963f4c2-fab1-4b97-b519-3c7a12008de5" containerName="payload-processing"
Sep 16 14:36:59.380016 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379853    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d01a563-84e9-4c4b-bd5a-5a1b120929ca" containerName="revoke-keys"
Sep 16 14:36:59.380016 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379868    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c963f4c2-fab1-4b97-b519-3c7a12008de5" containerName="payload-processing"
Sep 16 14:36:59.380016 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.379882    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="acca96f1-fa3e-4350-92fa-aa3f074e75e0" containerName="main"
Sep 16 14:36:59.383206 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.383185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"
Sep 16 14:36:59.386097 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.386077    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:36:59.386211 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.386077    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:36:59.394292 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.394269    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"]
Sep 16 14:36:59.506735 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.506704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc94754d-47bf-4b2b-bc0e-46db93008a5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5\" (UID: \"bc94754d-47bf-4b2b-bc0e-46db93008a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"
Sep 16 14:36:59.607623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.607590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc94754d-47bf-4b2b-bc0e-46db93008a5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5\" (UID: \"bc94754d-47bf-4b2b-bc0e-46db93008a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"
Sep 16 14:36:59.608258 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.608240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc94754d-47bf-4b2b-bc0e-46db93008a5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5\" (UID: \"bc94754d-47bf-4b2b-bc0e-46db93008a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"
Sep 16 14:36:59.693606 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.693525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"
Sep 16 14:36:59.967178 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:36:59.967154    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"]
Sep 16 14:36:59.969274 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:36:59.969249    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc94754d_47bf_4b2b_bc0e_46db93008a5c.slice/crio-a51c728a0611d5e4fdc833172d3525b172ea959d2823d64c159ccc5d4a16ce6e WatchSource:0}: Error finding container a51c728a0611d5e4fdc833172d3525b172ea959d2823d64c159ccc5d4a16ce6e: Status 404 returned error can't find the container with id a51c728a0611d5e4fdc833172d3525b172ea959d2823d64c159ccc5d4a16ce6e
Sep 16 14:37:00.517366 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:00.517336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5" event={"ID":"bc94754d-47bf-4b2b-bc0e-46db93008a5c","Type":"ContainerStarted","Data":"c2cca6b59c32823785c6ab72cf29fe77fd01b46c9d9ce0d33042069698854681"}
Sep 16 14:37:00.517545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:00.517387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5" event={"ID":"bc94754d-47bf-4b2b-bc0e-46db93008a5c","Type":"ContainerStarted","Data":"a51c728a0611d5e4fdc833172d3525b172ea959d2823d64c159ccc5d4a16ce6e"}
Sep 16 14:37:00.537362 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:00.537308    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5" podStartSLOduration=1.537289437 podStartE2EDuration="1.537289437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:37:00.535849655 +0000 UTC m=+1951.023859732" watchObservedRunningTime="2026-09-16 14:37:00.537289437 +0000 UTC m=+1951.025299514"
Sep 16 14:37:00.545233 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:00.545204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:01.524577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:01.524539    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc94754d-47bf-4b2b-bc0e-46db93008a5c" containerID="c2cca6b59c32823785c6ab72cf29fe77fd01b46c9d9ce0d33042069698854681" exitCode=0
Sep 16 14:37:01.525026 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:01.524580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5" event={"ID":"bc94754d-47bf-4b2b-bc0e-46db93008a5c","Type":"ContainerDied","Data":"c2cca6b59c32823785c6ab72cf29fe77fd01b46c9d9ce0d33042069698854681"}
Sep 16 14:37:01.586291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:01.586258    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:02.504456 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:02.504413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:02.663255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:02.663230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"
Sep 16 14:37:02.837861 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:02.837740    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc94754d-47bf-4b2b-bc0e-46db93008a5c-maas-api-service-ca\") pod \"bc94754d-47bf-4b2b-bc0e-46db93008a5c\" (UID: \"bc94754d-47bf-4b2b-bc0e-46db93008a5c\") "
Sep 16 14:37:02.838123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:02.838100    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc94754d-47bf-4b2b-bc0e-46db93008a5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bc94754d-47bf-4b2b-bc0e-46db93008a5c" (UID: "bc94754d-47bf-4b2b-bc0e-46db93008a5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:02.940210 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:02.940171    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc94754d-47bf-4b2b-bc0e-46db93008a5c-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:03.537500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:03.537462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5" event={"ID":"bc94754d-47bf-4b2b-bc0e-46db93008a5c","Type":"ContainerDied","Data":"a51c728a0611d5e4fdc833172d3525b172ea959d2823d64c159ccc5d4a16ce6e"}
Sep 16 14:37:03.537693 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:03.537516    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a51c728a0611d5e4fdc833172d3525b172ea959d2823d64c159ccc5d4a16ce6e"
Sep 16 14:37:03.537693 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:03.537474    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"
Sep 16 14:37:10.286818 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.286778    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:10.338641 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.338605    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"]
Sep 16 14:37:10.338942 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.338908    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr" podUID="72125b19-c7ae-4f80-8d62-b0cdd93cab30" containerName="payload-processing" containerID="cri-o://0e9b5fc347573a2b35f9a94c4a83c1b6a1611e6991612393665b9521b9bdf1fd" gracePeriod=30
Sep 16 14:37:10.342269 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.342241    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"]
Sep 16 14:37:10.342562 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.342533    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd" podUID="ae9ac34b-2021-4b54-8af7-50dd483d1e0f" containerName="payload-pre-processing" containerID="cri-o://c23ccacc34aa314b9294c06bf9482a01cc458be6f820b4fd973b8d9eb71a8780" gracePeriod=30
Sep 16 14:37:10.417585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.417549    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:10.512835 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.512381    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:10.578862 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.578822    2580 generic.go:356] "Generic (PLEG): container finished" podID="ae9ac34b-2021-4b54-8af7-50dd483d1e0f" containerID="c23ccacc34aa314b9294c06bf9482a01cc458be6f820b4fd973b8d9eb71a8780" exitCode=0
Sep 16 14:37:10.579043 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.578922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd" event={"ID":"ae9ac34b-2021-4b54-8af7-50dd483d1e0f","Type":"ContainerDied","Data":"c23ccacc34aa314b9294c06bf9482a01cc458be6f820b4fd973b8d9eb71a8780"}
Sep 16 14:37:10.581010 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.580979    2580 generic.go:356] "Generic (PLEG): container finished" podID="72125b19-c7ae-4f80-8d62-b0cdd93cab30" containerID="0e9b5fc347573a2b35f9a94c4a83c1b6a1611e6991612393665b9521b9bdf1fd" exitCode=0
Sep 16 14:37:10.581160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.581036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr" event={"ID":"72125b19-c7ae-4f80-8d62-b0cdd93cab30","Type":"ContainerDied","Data":"0e9b5fc347573a2b35f9a94c4a83c1b6a1611e6991612393665b9521b9bdf1fd"}
Sep 16 14:37:10.605533 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.605506    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:37:10.619490 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.619452    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:10.625109 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.625083    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"]
Sep 16 14:37:10.629858 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.629804    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:37:10.638412 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.638384    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7eeba8-zxmj5"]
Sep 16 14:37:10.713465 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.713428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72125b19-c7ae-4f80-8d62-b0cdd93cab30-plugins-config-volume\") pod \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") "
Sep 16 14:37:10.713680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.713516    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g897\" (UniqueName: \"kubernetes.io/projected/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-kube-api-access-4g897\") pod \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") "
Sep 16 14:37:10.713680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.713533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-plugins-config-volume\") pod \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\" (UID: \"ae9ac34b-2021-4b54-8af7-50dd483d1e0f\") "
Sep 16 14:37:10.713680 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.713550    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rrnw\" (UniqueName: \"kubernetes.io/projected/72125b19-c7ae-4f80-8d62-b0cdd93cab30-kube-api-access-8rrnw\") pod \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\" (UID: \"72125b19-c7ae-4f80-8d62-b0cdd93cab30\") "
Sep 16 14:37:10.713911 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.713883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72125b19-c7ae-4f80-8d62-b0cdd93cab30-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72125b19-c7ae-4f80-8d62-b0cdd93cab30" (UID: "72125b19-c7ae-4f80-8d62-b0cdd93cab30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:10.713980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.713946    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae9ac34b-2021-4b54-8af7-50dd483d1e0f" (UID: "ae9ac34b-2021-4b54-8af7-50dd483d1e0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:10.715798 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.715775    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-kube-api-access-4g897" (OuterVolumeSpecName: "kube-api-access-4g897") pod "ae9ac34b-2021-4b54-8af7-50dd483d1e0f" (UID: "ae9ac34b-2021-4b54-8af7-50dd483d1e0f"). InnerVolumeSpecName "kube-api-access-4g897". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:10.715887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.715836    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72125b19-c7ae-4f80-8d62-b0cdd93cab30-kube-api-access-8rrnw" (OuterVolumeSpecName: "kube-api-access-8rrnw") pod "72125b19-c7ae-4f80-8d62-b0cdd93cab30" (UID: "72125b19-c7ae-4f80-8d62-b0cdd93cab30"). InnerVolumeSpecName "kube-api-access-8rrnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:10.719882 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.719850    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:10.815271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.815179    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g897\" (UniqueName: \"kubernetes.io/projected/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-kube-api-access-4g897\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:10.815271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.815208    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae9ac34b-2021-4b54-8af7-50dd483d1e0f-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:10.815271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.815219    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rrnw\" (UniqueName: \"kubernetes.io/projected/72125b19-c7ae-4f80-8d62-b0cdd93cab30-kube-api-access-8rrnw\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:10.815271 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:10.815229    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72125b19-c7ae-4f80-8d62-b0cdd93cab30-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:11.033521 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.033484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:11.216836 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.216804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:11.428502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.428462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:11.523689 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.523586    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:11.588099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.588068    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"
Sep 16 14:37:11.588317 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.588114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd" event={"ID":"ae9ac34b-2021-4b54-8af7-50dd483d1e0f","Type":"ContainerDied","Data":"06a70ab2a228a7ebd04f961ad8015bcca4863171cc838ee09e563b98d1b45121"}
Sep 16 14:37:11.588317 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.588204    2580 scope.go:117] "RemoveContainer" containerID="c23ccacc34aa314b9294c06bf9482a01cc458be6f820b4fd973b8d9eb71a8780"
Sep 16 14:37:11.589684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.589496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr" event={"ID":"72125b19-c7ae-4f80-8d62-b0cdd93cab30","Type":"ContainerDied","Data":"5e665e8dd837b6af3fb08fa0a98ee96d2d2d0cdb736158e7092f246cc7fbb1a9"}
Sep 16 14:37:11.589684 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.589500    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"
Sep 16 14:37:11.601174 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.601153    2580 scope.go:117] "RemoveContainer" containerID="0e9b5fc347573a2b35f9a94c4a83c1b6a1611e6991612393665b9521b9bdf1fd"
Sep 16 14:37:11.650071 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.650034    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"]
Sep 16 14:37:11.660650 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.660621    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7eeba8-77dff58446-hj2kd"]
Sep 16 14:37:11.691182 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.691132    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"]
Sep 16 14:37:11.708560 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:11.708529    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7eeba8-75c6f498c9-slxxr"]
Sep 16 14:37:12.005976 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:12.005941    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72125b19-c7ae-4f80-8d62-b0cdd93cab30" path="/var/lib/kubelet/pods/72125b19-c7ae-4f80-8d62-b0cdd93cab30/volumes"
Sep 16 14:37:12.006336 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:12.006322    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae9ac34b-2021-4b54-8af7-50dd483d1e0f" path="/var/lib/kubelet/pods/ae9ac34b-2021-4b54-8af7-50dd483d1e0f/volumes"
Sep 16 14:37:12.006619 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:12.006607    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc94754d-47bf-4b2b-bc0e-46db93008a5c" path="/var/lib/kubelet/pods/bc94754d-47bf-4b2b-bc0e-46db93008a5c/volumes"
Sep 16 14:37:12.438097 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:12.438004    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:14.188646 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:14.188607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:37:17.903801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:17.903761    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:18.017454 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.017420    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:18.169816 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.169734    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"]
Sep 16 14:37:18.169985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.169936    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy" containerID="cri-o://341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec" gracePeriod=30
Sep 16 14:37:18.206810 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.206753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:18.555934 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.555898    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"]
Sep 16 14:37:18.556347 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556334    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc94754d-47bf-4b2b-bc0e-46db93008a5c" containerName="revoke-keys"
Sep 16 14:37:18.556406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556348    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc94754d-47bf-4b2b-bc0e-46db93008a5c" containerName="revoke-keys"
Sep 16 14:37:18.556406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556370    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72125b19-c7ae-4f80-8d62-b0cdd93cab30" containerName="payload-processing"
Sep 16 14:37:18.556406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556376    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="72125b19-c7ae-4f80-8d62-b0cdd93cab30" containerName="payload-processing"
Sep 16 14:37:18.556406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556394    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae9ac34b-2021-4b54-8af7-50dd483d1e0f" containerName="payload-pre-processing"
Sep 16 14:37:18.556406 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556403    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae9ac34b-2021-4b54-8af7-50dd483d1e0f" containerName="payload-pre-processing"
Sep 16 14:37:18.556567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556463    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc94754d-47bf-4b2b-bc0e-46db93008a5c" containerName="revoke-keys"
Sep 16 14:37:18.556567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556474    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="72125b19-c7ae-4f80-8d62-b0cdd93cab30" containerName="payload-processing"
Sep 16 14:37:18.556567 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.556481    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae9ac34b-2021-4b54-8af7-50dd483d1e0f" containerName="payload-pre-processing"
Sep 16 14:37:18.559395 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.559375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"
Sep 16 14:37:18.564972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.564942    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:37:18.564972 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.564954    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:37:18.576285 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.576258    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"]
Sep 16 14:37:18.589202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.589162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29367252-8f62-4415-ba62-67ec90b050aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r\" (UID: \"29367252-8f62-4415-ba62-67ec90b050aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"
Sep 16 14:37:18.690435 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.690397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29367252-8f62-4415-ba62-67ec90b050aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r\" (UID: \"29367252-8f62-4415-ba62-67ec90b050aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"
Sep 16 14:37:18.691041 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.691019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29367252-8f62-4415-ba62-67ec90b050aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r\" (UID: \"29367252-8f62-4415-ba62-67ec90b050aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"
Sep 16 14:37:18.870225 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.870123    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"
Sep 16 14:37:18.934425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:18.934385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:19.150903 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:19.150869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"]
Sep 16 14:37:19.151691 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:37:19.151668    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29367252_8f62_4415_ba62_67ec90b050aa.slice/crio-addd64dd4b419b0c55502ea272be582721709b939751f448136000eec1fe01ab WatchSource:0}: Error finding container addd64dd4b419b0c55502ea272be582721709b939751f448136000eec1fe01ab: Status 404 returned error can't find the container with id addd64dd4b419b0c55502ea272be582721709b939751f448136000eec1fe01ab
Sep 16 14:37:19.637411 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:19.637374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r" event={"ID":"29367252-8f62-4415-ba62-67ec90b050aa","Type":"ContainerStarted","Data":"92a0e48682dc7b758569f342c9427e263e3787321925712630ccdcb5002eecbc"}
Sep 16 14:37:19.637611 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:19.637435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r" event={"ID":"29367252-8f62-4415-ba62-67ec90b050aa","Type":"ContainerStarted","Data":"addd64dd4b419b0c55502ea272be582721709b939751f448136000eec1fe01ab"}
Sep 16 14:37:19.666801 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:19.666694    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r" podStartSLOduration=1.6666770199999998 podStartE2EDuration="1.66667702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:37:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:37:19.665468143 +0000 UTC m=+1970.153478220" watchObservedRunningTime="2026-09-16 14:37:19.66667702 +0000 UTC m=+1970.154687097"
Sep 16 14:37:20.039677 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:20.039643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:20.643831 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:20.643794    2580 generic.go:356] "Generic (PLEG): container finished" podID="29367252-8f62-4415-ba62-67ec90b050aa" containerID="92a0e48682dc7b758569f342c9427e263e3787321925712630ccdcb5002eecbc" exitCode=0
Sep 16 14:37:20.644028 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:20.643862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r" event={"ID":"29367252-8f62-4415-ba62-67ec90b050aa","Type":"ContainerDied","Data":"92a0e48682dc7b758569f342c9427e263e3787321925712630ccdcb5002eecbc"}
Sep 16 14:37:21.002959 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:21.002905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:21.785938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:21.785914    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"
Sep 16 14:37:21.823006 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:21.822976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29367252-8f62-4415-ba62-67ec90b050aa-maas-api-service-ca\") pod \"29367252-8f62-4415-ba62-67ec90b050aa\" (UID: \"29367252-8f62-4415-ba62-67ec90b050aa\") "
Sep 16 14:37:21.823320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:21.823298    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29367252-8f62-4415-ba62-67ec90b050aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29367252-8f62-4415-ba62-67ec90b050aa" (UID: "29367252-8f62-4415-ba62-67ec90b050aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:21.924828 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:21.924732    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29367252-8f62-4415-ba62-67ec90b050aa-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:22.655837 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:22.655760    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"
Sep 16 14:37:22.655837 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:22.655786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r" event={"ID":"29367252-8f62-4415-ba62-67ec90b050aa","Type":"ContainerDied","Data":"addd64dd4b419b0c55502ea272be582721709b939751f448136000eec1fe01ab"}
Sep 16 14:37:22.655837 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:22.655834    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="addd64dd4b419b0c55502ea272be582721709b939751f448136000eec1fe01ab"
Sep 16 14:37:23.427229 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.427201    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:37:23.542260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542221    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-data\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542267    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-istio-token\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542284    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-envoy\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542352    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-certs\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542373    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b28b172-88cb-4d69-8598-29720071a0a9-istiod-ca-cert\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542411    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b28b172-88cb-4d69-8598-29720071a0a9-istio-podinfo\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542452    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-credential-socket\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542673    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:23.542770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w998z\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-kube-api-access-w998z\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b28b172-88cb-4d69-8598-29720071a0a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:23.542889 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542859    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-socket\") pod \"0b28b172-88cb-4d69-8598-29720071a0a9\" (UID: \"0b28b172-88cb-4d69-8598-29720071a0a9\") "
Sep 16 14:37:23.542995 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542874    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:23.542995 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.542891    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:23.543218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.543197    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:23.543360 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.543341    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.543427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.543364    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b28b172-88cb-4d69-8598-29720071a0a9-istiod-ca-cert\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.543427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.543377    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-credential-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.543427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.543392    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-workload-socket\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.543427 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.543405    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-data\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.544770 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.544744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:23.544883 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.544854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-kube-api-access-w998z" (OuterVolumeSpecName: "kube-api-access-w998z") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "kube-api-access-w998z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:23.545032 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.545011    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:23.545032 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.545008    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b28b172-88cb-4d69-8598-29720071a0a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b28b172-88cb-4d69-8598-29720071a0a9" (UID: "0b28b172-88cb-4d69-8598-29720071a0a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:37:23.644904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.644863    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b28b172-88cb-4d69-8598-29720071a0a9-istio-podinfo\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.644904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.644896    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w998z\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-kube-api-access-w998z\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.644904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.644908    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b28b172-88cb-4d69-8598-29720071a0a9-istio-token\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.644904 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.644916    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b28b172-88cb-4d69-8598-29720071a0a9-istio-envoy\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:23.663108 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.663073    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b28b172-88cb-4d69-8598-29720071a0a9" containerID="341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec" exitCode=0
Sep 16 14:37:23.663332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.663131    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"
Sep 16 14:37:23.663332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.663168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" event={"ID":"0b28b172-88cb-4d69-8598-29720071a0a9","Type":"ContainerDied","Data":"341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec"}
Sep 16 14:37:23.663332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.663228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df" event={"ID":"0b28b172-88cb-4d69-8598-29720071a0a9","Type":"ContainerDied","Data":"028387bd41db41f7a132f51141311207e08c04aaaab31be0e02fa9d6d40899ee"}
Sep 16 14:37:23.663332 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.663253    2580 scope.go:117] "RemoveContainer" containerID="341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec"
Sep 16 14:37:23.673213 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.673197    2580 scope.go:117] "RemoveContainer" containerID="e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a"
Sep 16 14:37:23.681993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.681975    2580 scope.go:117] "RemoveContainer" containerID="341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec"
Sep 16 14:37:23.682304 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:37:23.682284    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec\": container with ID starting with 341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec not found: ID does not exist" containerID="341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec"
Sep 16 14:37:23.682359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.682314    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec"} err="failed to get container status \"341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec\": rpc error: code = NotFound desc = could not find container \"341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec\": container with ID starting with 341f237191c7998a7c4625b956a7e3e3e33c6cb685acda7a779b24dc2ee724ec not found: ID does not exist"
Sep 16 14:37:23.682359 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.682330    2580 scope.go:117] "RemoveContainer" containerID="e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a"
Sep 16 14:37:23.682553 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:37:23.682535    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a\": container with ID starting with e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a not found: ID does not exist" containerID="e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a"
Sep 16 14:37:23.682590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.682558    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a"} err="failed to get container status \"e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a\": rpc error: code = NotFound desc = could not find container \"e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a\": container with ID starting with e66e11445e4c0c607e5a169bec6a0290cdbd0f2b7b5cc1132586924ceb0d697a not found: ID does not exist"
Sep 16 14:37:23.695358 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.695286    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"]
Sep 16 14:37:23.697708 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:23.697688    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-7eeba8-openshift-default-d848d47fb-8t5df"]
Sep 16 14:37:24.006002 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:24.005968    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" path="/var/lib/kubelet/pods/0b28b172-88cb-4d69-8598-29720071a0a9/volumes"
Sep 16 14:37:29.269704 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.269666    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"]
Sep 16 14:37:29.276953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.276926    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-50b57c-29826150-fxpl8"]
Sep 16 14:37:29.348055 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.348018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:29.378018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.377984    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"]
Sep 16 14:37:29.378256 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.378225    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw" podUID="69b91b56-ada5-433b-8cbc-d56c84e6a775" containerName="payload-processing" containerID="cri-o://c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66" gracePeriod=30
Sep 16 14:37:29.456085 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.456044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:29.631697 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.631653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:29.633850 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.633832    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:37:29.696967 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.696936    2580 generic.go:356] "Generic (PLEG): container finished" podID="69b91b56-ada5-433b-8cbc-d56c84e6a775" containerID="c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66" exitCode=0
Sep 16 14:37:29.697157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.696982    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"
Sep 16 14:37:29.697157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.697025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw" event={"ID":"69b91b56-ada5-433b-8cbc-d56c84e6a775","Type":"ContainerDied","Data":"c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66"}
Sep 16 14:37:29.697157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.697077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw" event={"ID":"69b91b56-ada5-433b-8cbc-d56c84e6a775","Type":"ContainerDied","Data":"bfbcdee10615179b3b57a0c1ded8b7cada0723352ea0127ab1d92e18afa1dacd"}
Sep 16 14:37:29.697157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.697103    2580 scope.go:117] "RemoveContainer" containerID="c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66"
Sep 16 14:37:29.705898 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.705884    2580 scope.go:117] "RemoveContainer" containerID="c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66"
Sep 16 14:37:29.706220 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.706204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dskp\" (UniqueName: \"kubernetes.io/projected/69b91b56-ada5-433b-8cbc-d56c84e6a775-kube-api-access-6dskp\") pod \"69b91b56-ada5-433b-8cbc-d56c84e6a775\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") "
Sep 16 14:37:29.706289 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:37:29.706219    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66\": container with ID starting with c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66 not found: ID does not exist" containerID="c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66"
Sep 16 14:37:29.706289 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.706250    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66"} err="failed to get container status \"c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66\": rpc error: code = NotFound desc = could not find container \"c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66\": container with ID starting with c47b617e20451c18c4f9dd61c87feb010de02f710e76e4b88dcdbde3fee70b66 not found: ID does not exist"
Sep 16 14:37:29.706360 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.706299    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b91b56-ada5-433b-8cbc-d56c84e6a775-plugins-config-volume\") pod \"69b91b56-ada5-433b-8cbc-d56c84e6a775\" (UID: \"69b91b56-ada5-433b-8cbc-d56c84e6a775\") "
Sep 16 14:37:29.706743 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.706713    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69b91b56-ada5-433b-8cbc-d56c84e6a775-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69b91b56-ada5-433b-8cbc-d56c84e6a775" (UID: "69b91b56-ada5-433b-8cbc-d56c84e6a775"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:29.708149 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.708118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69b91b56-ada5-433b-8cbc-d56c84e6a775-kube-api-access-6dskp" (OuterVolumeSpecName: "kube-api-access-6dskp") pod "69b91b56-ada5-433b-8cbc-d56c84e6a775" (UID: "69b91b56-ada5-433b-8cbc-d56c84e6a775"). InnerVolumeSpecName "kube-api-access-6dskp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:29.735713 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.735677    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"]
Sep 16 14:37:29.737887 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.737861    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-50b57c-rhb4r"]
Sep 16 14:37:29.808018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.807933    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dskp\" (UniqueName: \"kubernetes.io/projected/69b91b56-ada5-433b-8cbc-d56c84e6a775-kube-api-access-6dskp\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:29.808018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.807962    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69b91b56-ada5-433b-8cbc-d56c84e6a775-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:29.874309 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:29.874275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:30.005918 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:30.005886    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29367252-8f62-4415-ba62-67ec90b050aa" path="/var/lib/kubelet/pods/29367252-8f62-4415-ba62-67ec90b050aa/volumes"
Sep 16 14:37:30.006355 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:30.006335    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6145853-ef62-4e27-b8ad-362437da0d94" path="/var/lib/kubelet/pods/d6145853-ef62-4e27-b8ad-362437da0d94/volumes"
Sep 16 14:37:30.028646 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:30.028612    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"]
Sep 16 14:37:30.036058 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:30.036025    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-50b57c-9bbc6665c-q4pqw"]
Sep 16 14:37:30.252370 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:30.252337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:30.352036 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:30.352001    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:30.479108 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:30.479072    2580 scope.go:117] "RemoveContainer" containerID="5076a2614873b32c3e657d15f070362a205a6a9aef79f0412f4186a92359f37f"
Sep 16 14:37:32.006001 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:32.005969    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69b91b56-ada5-433b-8cbc-d56c84e6a775" path="/var/lib/kubelet/pods/69b91b56-ada5-433b-8cbc-d56c84e6a775/volumes"
Sep 16 14:37:32.375050 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:32.374962    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:37:34.000672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:34.000637    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:37:36.679657 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:36.679578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:36.795789 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:36.795755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:37.010688 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.010656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:37.406569 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.406472    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"]
Sep 16 14:37:37.406985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.406965    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69b91b56-ada5-433b-8cbc-d56c84e6a775" containerName="payload-processing"
Sep 16 14:37:37.407086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.406988    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="69b91b56-ada5-433b-8cbc-d56c84e6a775" containerName="payload-processing"
Sep 16 14:37:37.407086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407004    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy"
Sep 16 14:37:37.407086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407012    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy"
Sep 16 14:37:37.407086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407023    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy"
Sep 16 14:37:37.407086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407032    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy"
Sep 16 14:37:37.407086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407078    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29367252-8f62-4415-ba62-67ec90b050aa" containerName="revoke-keys"
Sep 16 14:37:37.407086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407086    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29367252-8f62-4415-ba62-67ec90b050aa" containerName="revoke-keys"
Sep 16 14:37:37.407467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407197    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="69b91b56-ada5-433b-8cbc-d56c84e6a775" containerName="payload-processing"
Sep 16 14:37:37.407467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407210    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29367252-8f62-4415-ba62-67ec90b050aa" containerName="revoke-keys"
Sep 16 14:37:37.407467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407223    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy"
Sep 16 14:37:37.407467 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.407233    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b28b172-88cb-4d69-8598-29720071a0a9" containerName="istio-proxy"
Sep 16 14:37:37.410391 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.410362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"
Sep 16 14:37:37.416013 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.415990    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:37:37.416169 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.416103    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:37:37.426643 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.426612    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"]
Sep 16 14:37:37.595527 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.595481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64f8a3a-fa82-4338-bacb-e70674f31fdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2\" (UID: \"b64f8a3a-fa82-4338-bacb-e70674f31fdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"
Sep 16 14:37:37.696418 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.696317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64f8a3a-fa82-4338-bacb-e70674f31fdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2\" (UID: \"b64f8a3a-fa82-4338-bacb-e70674f31fdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"
Sep 16 14:37:37.696946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.696924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64f8a3a-fa82-4338-bacb-e70674f31fdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2\" (UID: \"b64f8a3a-fa82-4338-bacb-e70674f31fdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"
Sep 16 14:37:37.721274 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.721243    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"
Sep 16 14:37:37.983030 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.982994    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:37.998051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:37.998028    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"]
Sep 16 14:37:37.999874 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:37:37.999846    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64f8a3a_fa82_4338_bacb_e70674f31fdb.slice/crio-e0954fb51fb13ad18a8f3009324c37661b8647ca4463881e76730c34fb33978d WatchSource:0}: Error finding container e0954fb51fb13ad18a8f3009324c37661b8647ca4463881e76730c34fb33978d: Status 404 returned error can't find the container with id e0954fb51fb13ad18a8f3009324c37661b8647ca4463881e76730c34fb33978d
Sep 16 14:37:38.749750 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:38.749709    2580 generic.go:356] "Generic (PLEG): container finished" podID="b64f8a3a-fa82-4338-bacb-e70674f31fdb" containerID="c58e67b3756acf7f8dd99d295508e98378a003efc44ddb4a22580c5ff810aa7a" exitCode=0
Sep 16 14:37:38.750151 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:38.749759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2" event={"ID":"b64f8a3a-fa82-4338-bacb-e70674f31fdb","Type":"ContainerDied","Data":"c58e67b3756acf7f8dd99d295508e98378a003efc44ddb4a22580c5ff810aa7a"}
Sep 16 14:37:38.750151 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:38.749812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2" event={"ID":"b64f8a3a-fa82-4338-bacb-e70674f31fdb","Type":"ContainerStarted","Data":"e0954fb51fb13ad18a8f3009324c37661b8647ca4463881e76730c34fb33978d"}
Sep 16 14:37:39.896829 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:39.896804    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"
Sep 16 14:37:39.915255 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:39.915222    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64f8a3a-fa82-4338-bacb-e70674f31fdb-maas-api-service-ca\") pod \"b64f8a3a-fa82-4338-bacb-e70674f31fdb\" (UID: \"b64f8a3a-fa82-4338-bacb-e70674f31fdb\") "
Sep 16 14:37:39.915600 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:39.915572    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b64f8a3a-fa82-4338-bacb-e70674f31fdb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b64f8a3a-fa82-4338-bacb-e70674f31fdb" (UID: "b64f8a3a-fa82-4338-bacb-e70674f31fdb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:39.915766 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:39.915693    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64f8a3a-fa82-4338-bacb-e70674f31fdb-maas-api-service-ca\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:40.765586 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:40.765557    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"
Sep 16 14:37:40.765760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:40.765598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2" event={"ID":"b64f8a3a-fa82-4338-bacb-e70674f31fdb","Type":"ContainerDied","Data":"e0954fb51fb13ad18a8f3009324c37661b8647ca4463881e76730c34fb33978d"}
Sep 16 14:37:40.765760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:40.765641    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0954fb51fb13ad18a8f3009324c37661b8647ca4463881e76730c34fb33978d"
Sep 16 14:37:48.188437 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.188404    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"]
Sep 16 14:37:48.188945 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.188854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:48.191398 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.191375    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f72402-29826150-bd5nd"]
Sep 16 14:37:48.223217 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.223181    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"]
Sep 16 14:37:48.223486 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.223460    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4" podUID="44de50d6-2fd6-4e0a-bbca-fda842e39f6d" containerName="payload-pre-processing" containerID="cri-o://f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a" gracePeriod=30
Sep 16 14:37:48.295515 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.295477    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:48.390822 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.390782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:48.439991 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.439916    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"]
Sep 16 14:37:48.445256 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.445209    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f72402-g4ph2"]
Sep 16 14:37:48.476900 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.476879    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:37:48.505577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.505517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:48.582634 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.582596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx7dn\" (UniqueName: \"kubernetes.io/projected/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-kube-api-access-hx7dn\") pod \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") "
Sep 16 14:37:48.582836 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.582714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-plugins-config-volume\") pod \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\" (UID: \"44de50d6-2fd6-4e0a-bbca-fda842e39f6d\") "
Sep 16 14:37:48.583104 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.583078    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44de50d6-2fd6-4e0a-bbca-fda842e39f6d" (UID: "44de50d6-2fd6-4e0a-bbca-fda842e39f6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:48.584867 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.584844    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-kube-api-access-hx7dn" (OuterVolumeSpecName: "kube-api-access-hx7dn") pod "44de50d6-2fd6-4e0a-bbca-fda842e39f6d" (UID: "44de50d6-2fd6-4e0a-bbca-fda842e39f6d"). InnerVolumeSpecName "kube-api-access-hx7dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:48.684279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.684244    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-plugins-config-volume\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:48.684279 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.684275    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx7dn\" (UniqueName: \"kubernetes.io/projected/44de50d6-2fd6-4e0a-bbca-fda842e39f6d-kube-api-access-hx7dn\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:37:48.808541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.808442    2580 generic.go:356] "Generic (PLEG): container finished" podID="44de50d6-2fd6-4e0a-bbca-fda842e39f6d" containerID="f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a" exitCode=0
Sep 16 14:37:48.808541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.808495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4" event={"ID":"44de50d6-2fd6-4e0a-bbca-fda842e39f6d","Type":"ContainerDied","Data":"f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a"}
Sep 16 14:37:48.808541 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.808502    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"
Sep 16 14:37:48.808864 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.808571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4" event={"ID":"44de50d6-2fd6-4e0a-bbca-fda842e39f6d","Type":"ContainerDied","Data":"46a55b6ed70b33339b4402f9b2751ce1368782af580a8ae7bf090c3f1c9da01a"}
Sep 16 14:37:48.808864 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.808601    2580 scope.go:117] "RemoveContainer" containerID="f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a"
Sep 16 14:37:48.818284 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.818265    2580 scope.go:117] "RemoveContainer" containerID="f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a"
Sep 16 14:37:48.818530 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:37:48.818510    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a\": container with ID starting with f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a not found: ID does not exist" containerID="f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a"
Sep 16 14:37:48.818576 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.818540    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a"} err="failed to get container status \"f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a\": rpc error: code = NotFound desc = could not find container \"f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a\": container with ID starting with f843a03ad32b73e7cee8e7943955759b17c3796bea200c4e561feede6ef74d3a not found: ID does not exist"
Sep 16 14:37:48.836899 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.836818    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"]
Sep 16 14:37:48.838873 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:48.838842    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f72402-548cc9c68b-ddbz4"]
Sep 16 14:37:49.150438 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:49.150338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:50.005953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:50.005919    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44de50d6-2fd6-4e0a-bbca-fda842e39f6d" path="/var/lib/kubelet/pods/44de50d6-2fd6-4e0a-bbca-fda842e39f6d/volumes"
Sep 16 14:37:50.006381 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:50.006286    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b64f8a3a-fa82-4338-bacb-e70674f31fdb" path="/var/lib/kubelet/pods/b64f8a3a-fa82-4338-bacb-e70674f31fdb/volumes"
Sep 16 14:37:50.006543 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:50.006530    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8e646a4-7fd6-424d-9500-2dfbfecd6281" path="/var/lib/kubelet/pods/d8e646a4-7fd6-424d-9500-2dfbfecd6281/volumes"
Sep 16 14:37:51.014297 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:51.014264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:37:55.257704 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:55.257666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:55.322842 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:55.322803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:55.542738 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:55.542637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:37:59.169701 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:37:59.169663    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:38:01.049690 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:38:01.049653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:38:12.000218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:38:12.000182    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:38:30.516003 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:38:30.515971    2580 scope.go:117] "RemoveContainer" containerID="8c819618a2c648dc5f9400877c0538d8ed942a1164ffc86dbffe229f3c5bd484"
Sep 16 14:38:42.629074 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:38:42.629031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:38:42.710998 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:38:42.710958    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:38:57.000478 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:38:57.000395    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:39:21.782720 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:39:21.782687    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:39:26.236425 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:39:26.236391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:39:28.272537 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:39:28.272500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:39:30.002722 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:39:30.002691    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:39:54.577295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:39:54.577262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:39:59.000198 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:39:59.000164    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:40:02.488016 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:02.487981    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:40:09.689200 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:09.689167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:40:18.483645 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:18.483608    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:40:30.548464 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:30.548352    2580 scope.go:117] "RemoveContainer" containerID="adaa68b9735c20004a9a77769dd7a3682fbf22d54fd6793675dd5d5bb85caa6d"
Sep 16 14:40:30.556923 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:30.556899    2580 scope.go:117] "RemoveContainer" containerID="3429657d31991ee08d631a4cb982f89c8e955e4917b7bd8dda79b1d74e744f6c"
Sep 16 14:40:33.480909 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:33.480872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:40:34.032616 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:34.032577    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:40:40.002373 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:40.002343    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:40:41.405927 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:40:41.405895    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:41:02.765737 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:41:02.765695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:41:06.000175 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:41:06.000117    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:41:30.582935 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:41:30.582903    2580 scope.go:117] "RemoveContainer" containerID="c79bcb7efeea4d0a766198e6ae217849e6e1a8489af021fdbf0549cefe2c2080"
Sep 16 14:41:30.591457 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:41:30.591436    2580 scope.go:117] "RemoveContainer" containerID="10d67981c6b6fb2b49cfa5f4b942625501fddc1b7d8837691b825a9ac0f21d16"
Sep 16 14:41:59.000079 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:41:59.000002    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:41:59.804107 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:41:59.804074    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:42:10.267491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:10.267459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:42:18.291041 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:18.291006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:42:25.000577 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:25.000541    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:42:26.867729 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:26.867690    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:42:30.614767 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:30.614735    2580 scope.go:117] "RemoveContainer" containerID="2c639fde32d90a573ea1216ee5721c69df698e2da950862fc66f07a4ba082c75"
Sep 16 14:42:30.622702 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:30.622685    2580 scope.go:117] "RemoveContainer" containerID="573ed4d1fbcb733d0d1834e9f5772370e72bc7297c190e5dce7fa06c360ace52"
Sep 16 14:42:36.508147 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.508098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"]
Sep 16 14:42:36.508590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.508520    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44de50d6-2fd6-4e0a-bbca-fda842e39f6d" containerName="payload-pre-processing"
Sep 16 14:42:36.508590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.508532    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="44de50d6-2fd6-4e0a-bbca-fda842e39f6d" containerName="payload-pre-processing"
Sep 16 14:42:36.508590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.508540    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64f8a3a-fa82-4338-bacb-e70674f31fdb" containerName="revoke-keys"
Sep 16 14:42:36.508590 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.508546    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64f8a3a-fa82-4338-bacb-e70674f31fdb" containerName="revoke-keys"
Sep 16 14:42:36.508823 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.508601    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="44de50d6-2fd6-4e0a-bbca-fda842e39f6d" containerName="payload-pre-processing"
Sep 16 14:42:36.508823 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.508614    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64f8a3a-fa82-4338-bacb-e70674f31fdb" containerName="revoke-keys"
Sep 16 14:42:36.511612 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.511594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.515594 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.515574    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-6f197787-kserve-self-signed-certs\""
Sep 16 14:42:36.537779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.537757    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"]
Sep 16 14:42:36.592985 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.592955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:42:36.668649 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.668612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-home\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.668841 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.668681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-dshm\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.668841 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.668735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8zfl\" (UniqueName: \"kubernetes.io/projected/3d091a01-cc92-4e6c-b87d-55136c605989-kube-api-access-w8zfl\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.668946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.668838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-tmp-dir\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.668946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.668888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-model-cache\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.668946 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.668923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d091a01-cc92-4e6c-b87d-55136c605989-tls-certs\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-tmp-dir\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-model-cache\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769516 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d091a01-cc92-4e6c-b87d-55136c605989-tls-certs\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769805 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-home\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769805 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-dshm\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769805 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8zfl\" (UniqueName: \"kubernetes.io/projected/3d091a01-cc92-4e6c-b87d-55136c605989-kube-api-access-w8zfl\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-tmp-dir\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-model-cache\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.769999 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.769982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-home\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.771897 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.771874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-dshm\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.772005 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.771959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d091a01-cc92-4e6c-b87d-55136c605989-tls-certs\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.788291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.788268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8zfl\" (UniqueName: \"kubernetes.io/projected/3d091a01-cc92-4e6c-b87d-55136c605989-kube-api-access-w8zfl\") pod \"e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") " pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:36.820099 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:36.820072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:37.109545 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:37.109521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"]
Sep 16 14:42:37.111918 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:42:37.111889    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d091a01_cc92_4e6c_b87d_55136c605989.slice/crio-77379be8c194917fdf9039a06bbe4dab7c21c05188e69f410ca38ea60f71b834 WatchSource:0}: Error finding container 77379be8c194917fdf9039a06bbe4dab7c21c05188e69f410ca38ea60f71b834: Status 404 returned error can't find the container with id 77379be8c194917fdf9039a06bbe4dab7c21c05188e69f410ca38ea60f71b834
Sep 16 14:42:37.274585 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:37.274552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp" event={"ID":"3d091a01-cc92-4e6c-b87d-55136c605989","Type":"ContainerStarted","Data":"46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4"}
Sep 16 14:42:37.274771 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:37.274595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp" event={"ID":"3d091a01-cc92-4e6c-b87d-55136c605989","Type":"ContainerStarted","Data":"77379be8c194917fdf9039a06bbe4dab7c21c05188e69f410ca38ea60f71b834"}
Sep 16 14:42:37.298022 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:37.297973    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp" podStartSLOduration=1.2979573979999999 podStartE2EDuration="1.297957398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:42:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:42:37.295390769 +0000 UTC m=+2287.783400859" watchObservedRunningTime="2026-09-16 14:42:37.297957398 +0000 UTC m=+2287.785967472"
Sep 16 14:42:39.572666 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:39.572632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:42:46.820820 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:46.820781    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:46.821461 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:46.820841    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:46.834109 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:46.834080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:47.336860 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:47.336834    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:42:59.770693 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.770657    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"]
Sep 16 14:42:59.774580 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.774555    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.777234 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.777209    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6f197787-kserve-self-signed-certs\""
Sep 16 14:42:59.783724 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.783705    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"]
Sep 16 14:42:59.870875 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.870846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:42:59.881935 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.881914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3144a39-6da3-4562-b7c6-cf41aa221062-tls-certs\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.882029 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.881956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-home\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.882103 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.882050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-model-cache\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.882187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.882105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-dshm\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.882187 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.882161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-tmp-dir\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.882296 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.882242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn2sb\" (UniqueName: \"kubernetes.io/projected/e3144a39-6da3-4562-b7c6-cf41aa221062-kube-api-access-hn2sb\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.982768 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.982736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3144a39-6da3-4562-b7c6-cf41aa221062-tls-certs\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.982926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.982783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-home\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.982926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.982822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-model-cache\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.982926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.982840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-dshm\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.982926 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.982858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-tmp-dir\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.983171 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.982990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn2sb\" (UniqueName: \"kubernetes.io/projected/e3144a39-6da3-4562-b7c6-cf41aa221062-kube-api-access-hn2sb\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.983342 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.983322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-tmp-dir\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.983404 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.983364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-model-cache\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.983404 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.983393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-home\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.985054 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.985032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-dshm\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.985374 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.985357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3144a39-6da3-4562-b7c6-cf41aa221062-tls-certs\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:42:59.991570 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:42:59.991553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn2sb\" (UniqueName: \"kubernetes.io/projected/e3144a39-6da3-4562-b7c6-cf41aa221062-kube-api-access-hn2sb\") pod \"e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") " pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:00.085584 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:00.085517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:00.357051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:00.357025    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"]
Sep 16 14:43:00.359121 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:43:00.359095    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3144a39_6da3_4562_b7c6_cf41aa221062.slice/crio-3823b5045ecae3f60b6557ad3aef170266fbbc958d289e532a0c869367bd0699 WatchSource:0}: Error finding container 3823b5045ecae3f60b6557ad3aef170266fbbc958d289e532a0c869367bd0699: Status 404 returned error can't find the container with id 3823b5045ecae3f60b6557ad3aef170266fbbc958d289e532a0c869367bd0699
Sep 16 14:43:00.391300 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:00.391273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm" event={"ID":"e3144a39-6da3-4562-b7c6-cf41aa221062","Type":"ContainerStarted","Data":"3823b5045ecae3f60b6557ad3aef170266fbbc958d289e532a0c869367bd0699"}
Sep 16 14:43:01.397846 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:01.397812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm" event={"ID":"e3144a39-6da3-4562-b7c6-cf41aa221062","Type":"ContainerStarted","Data":"9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de"}
Sep 16 14:43:01.416266 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:01.416221    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm" podStartSLOduration=2.416207219 podStartE2EDuration="2.416207219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:43:01.414978163 +0000 UTC m=+2311.902988238" watchObservedRunningTime="2026-09-16 14:43:01.416207219 +0000 UTC m=+2311.904217294"
Sep 16 14:43:02.775451 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:02.775415    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:43:10.085993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:10.085965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:10.085993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:10.086007    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:10.098494 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:10.098471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:10.456705 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:10.456674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:18.000669 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:18.000636    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:43:23.892783 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:23.892704    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"]
Sep 16 14:43:23.958018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:23.957986    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:43:25.567996 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:25.567964    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"]
Sep 16 14:43:25.601466 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:25.601434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:43:26.119724 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:26.119694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:43:30.645024 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:30.644991    2580 scope.go:117] "RemoveContainer" containerID="c2cca6b59c32823785c6ab72cf29fe77fd01b46c9d9ce0d33042069698854681"
Sep 16 14:43:30.653445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:30.653424    2580 scope.go:117] "RemoveContainer" containerID="92a0e48682dc7b758569f342c9427e263e3787321925712630ccdcb5002eecbc"
Sep 16 14:43:35.570062 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:35.570029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:43:38.911735 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:38.911675    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp" podUID="3d091a01-cc92-4e6c-b87d-55136c605989" containerName="main" containerID="cri-o://46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4" gracePeriod=45
Sep 16 14:43:39.161291 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.161269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:43:39.243041 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243016    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d091a01-cc92-4e6c-b87d-55136c605989-tls-certs\") pod \"3d091a01-cc92-4e6c-b87d-55136c605989\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") "
Sep 16 14:43:39.243237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243068    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-model-cache\") pod \"3d091a01-cc92-4e6c-b87d-55136c605989\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") "
Sep 16 14:43:39.243237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243100    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-home\") pod \"3d091a01-cc92-4e6c-b87d-55136c605989\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") "
Sep 16 14:43:39.243237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243166    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8zfl\" (UniqueName: \"kubernetes.io/projected/3d091a01-cc92-4e6c-b87d-55136c605989-kube-api-access-w8zfl\") pod \"3d091a01-cc92-4e6c-b87d-55136c605989\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") "
Sep 16 14:43:39.243237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243199    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-dshm\") pod \"3d091a01-cc92-4e6c-b87d-55136c605989\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") "
Sep 16 14:43:39.243237 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243218    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-tmp-dir\") pod \"3d091a01-cc92-4e6c-b87d-55136c605989\" (UID: \"3d091a01-cc92-4e6c-b87d-55136c605989\") "
Sep 16 14:43:39.243515 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d091a01-cc92-4e6c-b87d-55136c605989" (UID: "3d091a01-cc92-4e6c-b87d-55136c605989"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:39.243515 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-home" (OuterVolumeSpecName: "home") pod "3d091a01-cc92-4e6c-b87d-55136c605989" (UID: "3d091a01-cc92-4e6c-b87d-55136c605989"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:39.243623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243565    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:39.243623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d091a01-cc92-4e6c-b87d-55136c605989" (UID: "3d091a01-cc92-4e6c-b87d-55136c605989"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:39.243623 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.243582    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:39.245463 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.245435    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d091a01-cc92-4e6c-b87d-55136c605989-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d091a01-cc92-4e6c-b87d-55136c605989" (UID: "3d091a01-cc92-4e6c-b87d-55136c605989"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:43:39.245463 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.245438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-dshm" (OuterVolumeSpecName: "dshm") pod "3d091a01-cc92-4e6c-b87d-55136c605989" (UID: "3d091a01-cc92-4e6c-b87d-55136c605989"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:39.245627 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.245494    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d091a01-cc92-4e6c-b87d-55136c605989-kube-api-access-w8zfl" (OuterVolumeSpecName: "kube-api-access-w8zfl") pod "3d091a01-cc92-4e6c-b87d-55136c605989" (UID: "3d091a01-cc92-4e6c-b87d-55136c605989"). InnerVolumeSpecName "kube-api-access-w8zfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:43:39.344975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.344938    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:39.344975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.344968    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d091a01-cc92-4e6c-b87d-55136c605989-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:39.344975 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.344981    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d091a01-cc92-4e6c-b87d-55136c605989-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:39.345257 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.344995    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8zfl\" (UniqueName: \"kubernetes.io/projected/3d091a01-cc92-4e6c-b87d-55136c605989-kube-api-access-w8zfl\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:39.591089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.590988    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d091a01-cc92-4e6c-b87d-55136c605989" containerID="46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4" exitCode=0
Sep 16 14:43:39.591089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.591044    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"
Sep 16 14:43:39.591089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.591066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp" event={"ID":"3d091a01-cc92-4e6c-b87d-55136c605989","Type":"ContainerDied","Data":"46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4"}
Sep 16 14:43:39.591397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.591097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp" event={"ID":"3d091a01-cc92-4e6c-b87d-55136c605989","Type":"ContainerDied","Data":"77379be8c194917fdf9039a06bbe4dab7c21c05188e69f410ca38ea60f71b834"}
Sep 16 14:43:39.591397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.591115    2580 scope.go:117] "RemoveContainer" containerID="46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4"
Sep 16 14:43:39.601122 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.601106    2580 scope.go:117] "RemoveContainer" containerID="46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4"
Sep 16 14:43:39.601420 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:43:39.601403    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4\": container with ID starting with 46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4 not found: ID does not exist" containerID="46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4"
Sep 16 14:43:39.601488 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.601428    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4"} err="failed to get container status \"46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4\": rpc error: code = NotFound desc = could not find container \"46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4\": container with ID starting with 46cdbc2247cc7b4607ac64a81b6a17e37398fe4d4448e134bf7d1d7f9ac68fb4 not found: ID does not exist"
Sep 16 14:43:39.614885 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.614863    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"]
Sep 16 14:43:39.620753 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:39.620733    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-6f197787-kserve-5bd5d7d4f9-jjjpp"]
Sep 16 14:43:40.006327 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.006293    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d091a01-cc92-4e6c-b87d-55136c605989" path="/var/lib/kubelet/pods/3d091a01-cc92-4e6c-b87d-55136c605989/volumes"
Sep 16 14:43:40.586502 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.586468    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm" podUID="e3144a39-6da3-4562-b7c6-cf41aa221062" containerName="main" containerID="cri-o://9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de" gracePeriod=45
Sep 16 14:43:40.836458 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.836432    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:40.960511 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.960477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-dshm\") pod \"e3144a39-6da3-4562-b7c6-cf41aa221062\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") "
Sep 16 14:43:40.960672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.960556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn2sb\" (UniqueName: \"kubernetes.io/projected/e3144a39-6da3-4562-b7c6-cf41aa221062-kube-api-access-hn2sb\") pod \"e3144a39-6da3-4562-b7c6-cf41aa221062\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") "
Sep 16 14:43:40.960672 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.960635    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3144a39-6da3-4562-b7c6-cf41aa221062-tls-certs\") pod \"e3144a39-6da3-4562-b7c6-cf41aa221062\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") "
Sep 16 14:43:40.960752 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.960733    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-model-cache\") pod \"e3144a39-6da3-4562-b7c6-cf41aa221062\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") "
Sep 16 14:43:40.960790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.960780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-tmp-dir\") pod \"e3144a39-6da3-4562-b7c6-cf41aa221062\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") "
Sep 16 14:43:40.960832 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.960806    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-home\") pod \"e3144a39-6da3-4562-b7c6-cf41aa221062\" (UID: \"e3144a39-6da3-4562-b7c6-cf41aa221062\") "
Sep 16 14:43:40.961076 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.961007    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3144a39-6da3-4562-b7c6-cf41aa221062" (UID: "e3144a39-6da3-4562-b7c6-cf41aa221062"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:40.961249 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.961127    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3144a39-6da3-4562-b7c6-cf41aa221062" (UID: "e3144a39-6da3-4562-b7c6-cf41aa221062"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:40.961249 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.961167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-home" (OuterVolumeSpecName: "home") pod "e3144a39-6da3-4562-b7c6-cf41aa221062" (UID: "e3144a39-6da3-4562-b7c6-cf41aa221062"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:40.961249 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.961246    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-model-cache\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:40.962818 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.962795    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-dshm" (OuterVolumeSpecName: "dshm") pod "e3144a39-6da3-4562-b7c6-cf41aa221062" (UID: "e3144a39-6da3-4562-b7c6-cf41aa221062"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:43:40.962818 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.962807    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3144a39-6da3-4562-b7c6-cf41aa221062-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3144a39-6da3-4562-b7c6-cf41aa221062" (UID: "e3144a39-6da3-4562-b7c6-cf41aa221062"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:43:40.962956 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:40.962898    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3144a39-6da3-4562-b7c6-cf41aa221062-kube-api-access-hn2sb" (OuterVolumeSpecName: "kube-api-access-hn2sb") pod "e3144a39-6da3-4562-b7c6-cf41aa221062" (UID: "e3144a39-6da3-4562-b7c6-cf41aa221062"). InnerVolumeSpecName "kube-api-access-hn2sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:43:41.062315 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.062275    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-home\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:41.062315 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.062307    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-dshm\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:41.062315 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.062317    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn2sb\" (UniqueName: \"kubernetes.io/projected/e3144a39-6da3-4562-b7c6-cf41aa221062-kube-api-access-hn2sb\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:41.062781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.062328    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3144a39-6da3-4562-b7c6-cf41aa221062-tls-certs\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:41.062781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.062336    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3144a39-6da3-4562-b7c6-cf41aa221062-tmp-dir\") on node \"ip-10-0-137-34.ec2.internal\" DevicePath \"\""
Sep 16 14:43:41.603828 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.603790    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3144a39-6da3-4562-b7c6-cf41aa221062" containerID="9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de" exitCode=0
Sep 16 14:43:41.603980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.603853    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"
Sep 16 14:43:41.603980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.603877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm" event={"ID":"e3144a39-6da3-4562-b7c6-cf41aa221062","Type":"ContainerDied","Data":"9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de"}
Sep 16 14:43:41.603980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.603923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm" event={"ID":"e3144a39-6da3-4562-b7c6-cf41aa221062","Type":"ContainerDied","Data":"3823b5045ecae3f60b6557ad3aef170266fbbc958d289e532a0c869367bd0699"}
Sep 16 14:43:41.603980 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.603942    2580 scope.go:117] "RemoveContainer" containerID="9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de"
Sep 16 14:43:41.613413 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.613398    2580 scope.go:117] "RemoveContainer" containerID="9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de"
Sep 16 14:43:41.613699 ip-10-0-137-34 kubenswrapper[2580]: E0916 14:43:41.613679    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de\": container with ID starting with 9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de not found: ID does not exist" containerID="9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de"
Sep 16 14:43:41.613752 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.613705    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de"} err="failed to get container status \"9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de\": rpc error: code = NotFound desc = could not find container \"9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de\": container with ID starting with 9fb507789fc953df473e00f39ffd0423bbb4593a7bdc22c60a3085ea83a0c5de not found: ID does not exist"
Sep 16 14:43:41.625888 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.625862    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"]
Sep 16 14:43:41.630688 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:41.630666    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6f197787-kserve-66f5dd9d87-xlvbm"]
Sep 16 14:43:42.000118 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:42.000085    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:43:42.005998 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:42.005963    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3144a39-6da3-4562-b7c6-cf41aa221062" path="/var/lib/kubelet/pods/e3144a39-6da3-4562-b7c6-cf41aa221062/volumes"
Sep 16 14:43:44.996740 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:43:44.996704    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:44:30.678645 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:44:30.678612    2580 scope.go:117] "RemoveContainer" containerID="c58e67b3756acf7f8dd99d295508e98378a003efc44ddb4a22580c5ff810aa7a"
Sep 16 14:44:31.000854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:44:31.000822    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:45:00.132979 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.132900    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv"]
Sep 16 14:45:00.133469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.133312    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3144a39-6da3-4562-b7c6-cf41aa221062" containerName="main"
Sep 16 14:45:00.133469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.133325    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3144a39-6da3-4562-b7c6-cf41aa221062" containerName="main"
Sep 16 14:45:00.133469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.133341    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d091a01-cc92-4e6c-b87d-55136c605989" containerName="main"
Sep 16 14:45:00.133469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.133346    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d091a01-cc92-4e6c-b87d-55136c605989" containerName="main"
Sep 16 14:45:00.133469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.133402    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d091a01-cc92-4e6c-b87d-55136c605989" containerName="main"
Sep 16 14:45:00.133469 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.133411    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3144a39-6da3-4562-b7c6-cf41aa221062" containerName="main"
Sep 16 14:45:00.136462 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.136443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv"
Sep 16 14:45:00.139089 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.139068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:45:00.144356 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.144330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv"]
Sep 16 14:45:00.148111 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.148093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv"
Sep 16 14:45:00.427376 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:00.427348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv"]
Sep 16 14:45:00.429311 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:45:00.429283    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5b3025_38b8_4ad0_ad6f_34c4deaa2794.slice/crio-a43e340c7c051c74fa155e0c9c7a572466de488e91da2d4d858b1ff3ab86eeba WatchSource:0}: Error finding container a43e340c7c051c74fa155e0c9c7a572466de488e91da2d4d858b1ff3ab86eeba: Status 404 returned error can't find the container with id a43e340c7c051c74fa155e0c9c7a572466de488e91da2d4d858b1ff3ab86eeba
Sep 16 14:45:01.002818 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:01.002780    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv" event={"ID":"7d5b3025-38b8-4ad0-ad6f-34c4deaa2794","Type":"ContainerStarted","Data":"ac9fa0ecca2aa3bf92eda055ed02a03df22927afeaa6421ef070dc8dfb414ff0"}
Sep 16 14:45:01.002960 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:01.002850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv" event={"ID":"7d5b3025-38b8-4ad0-ad6f-34c4deaa2794","Type":"ContainerStarted","Data":"a43e340c7c051c74fa155e0c9c7a572466de488e91da2d4d858b1ff3ab86eeba"}
Sep 16 14:45:01.022489 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:01.022443    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv" podStartSLOduration=1.022425709 podStartE2EDuration="1.022425709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:45:01.022193967 +0000 UTC m=+2431.510204043" watchObservedRunningTime="2026-09-16 14:45:01.022425709 +0000 UTC m=+2431.510435800"
Sep 16 14:45:02.008202 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:02.008169    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d5b3025-38b8-4ad0-ad6f-34c4deaa2794" containerID="ac9fa0ecca2aa3bf92eda055ed02a03df22927afeaa6421ef070dc8dfb414ff0" exitCode=0
Sep 16 14:45:02.008596 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:02.008248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv" event={"ID":"7d5b3025-38b8-4ad0-ad6f-34c4deaa2794","Type":"ContainerDied","Data":"ac9fa0ecca2aa3bf92eda055ed02a03df22927afeaa6421ef070dc8dfb414ff0"}
Sep 16 14:45:03.146986 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:03.146955    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv"
Sep 16 14:45:04.018126 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:04.018100    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv"
Sep 16 14:45:04.018257 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:04.018125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-vfwsv" event={"ID":"7d5b3025-38b8-4ad0-ad6f-34c4deaa2794","Type":"ContainerDied","Data":"a43e340c7c051c74fa155e0c9c7a572466de488e91da2d4d858b1ff3ab86eeba"}
Sep 16 14:45:04.018257 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:04.018191    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a43e340c7c051c74fa155e0c9c7a572466de488e91da2d4d858b1ff3ab86eeba"
Sep 16 14:45:07.000605 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:07.000572    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:45:35.053298 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:35.053263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:45:47.000051 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:45:47.000011    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:46:33.000976 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:46:33.000940    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-wfbl9" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:46:51.473797 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:46:51.473764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:46:58.469630 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:46:58.469594    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:47:04.370817 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:04.370784    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:47:12.064859 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:12.064825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-crfbz"]
Sep 16 14:47:13.000760 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:13.000728    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-28hd7" secret="" err="secret \"payload-processing-dockercfg-ccsnl\" not found"
Sep 16 14:47:15.511241 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.511204    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-74mws/must-gather-mfpfv"]
Sep 16 14:47:15.511658 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.511631    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5b3025-38b8-4ad0-ad6f-34c4deaa2794" containerName="cleanup"
Sep 16 14:47:15.511658 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.511643    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5b3025-38b8-4ad0-ad6f-34c4deaa2794" containerName="cleanup"
Sep 16 14:47:15.511790 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.511716    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5b3025-38b8-4ad0-ad6f-34c4deaa2794" containerName="cleanup"
Sep 16 14:47:15.514854 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.514838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:15.517587 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.517559    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-74mws\"/\"openshift-service-ca.crt\""
Sep 16 14:47:15.517728 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.517691    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-74mws\"/\"default-dockercfg-cq4jd\""
Sep 16 14:47:15.517810 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.517795    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-74mws\"/\"kube-root-ca.crt\""
Sep 16 14:47:15.529934 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.529909    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74mws/must-gather-mfpfv"]
Sep 16 14:47:15.571908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.571873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:47:15.598032 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.597999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/82af91d8-9757-4089-abd5-228e7189a555-must-gather-output\") pod \"must-gather-mfpfv\" (UID: \"82af91d8-9757-4089-abd5-228e7189a555\") " pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:15.598218 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.598059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt6mh\" (UniqueName: \"kubernetes.io/projected/82af91d8-9757-4089-abd5-228e7189a555-kube-api-access-qt6mh\") pod \"must-gather-mfpfv\" (UID: \"82af91d8-9757-4089-abd5-228e7189a555\") " pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:15.699282 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.699248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/82af91d8-9757-4089-abd5-228e7189a555-must-gather-output\") pod \"must-gather-mfpfv\" (UID: \"82af91d8-9757-4089-abd5-228e7189a555\") " pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:15.699452 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.699300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6mh\" (UniqueName: \"kubernetes.io/projected/82af91d8-9757-4089-abd5-228e7189a555-kube-api-access-qt6mh\") pod \"must-gather-mfpfv\" (UID: \"82af91d8-9757-4089-abd5-228e7189a555\") " pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:15.699597 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.699570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/82af91d8-9757-4089-abd5-228e7189a555-must-gather-output\") pod \"must-gather-mfpfv\" (UID: \"82af91d8-9757-4089-abd5-228e7189a555\") " pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:15.709320 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.709298    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6mh\" (UniqueName: \"kubernetes.io/projected/82af91d8-9757-4089-abd5-228e7189a555-kube-api-access-qt6mh\") pod \"must-gather-mfpfv\" (UID: \"82af91d8-9757-4089-abd5-228e7189a555\") " pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:15.823808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:15.823721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74mws/must-gather-mfpfv"
Sep 16 14:47:16.095908 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:16.095873    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74mws/must-gather-mfpfv"]
Sep 16 14:47:16.098037 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:47:16.098007    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82af91d8_9757_4089_abd5_228e7189a555.slice/crio-0c2f0a938a570e3818192251b74eb5c6942a707a4d20d6dd1ce83c11c7d40fcd WatchSource:0}: Error finding container 0c2f0a938a570e3818192251b74eb5c6942a707a4d20d6dd1ce83c11c7d40fcd: Status 404 returned error can't find the container with id 0c2f0a938a570e3818192251b74eb5c6942a707a4d20d6dd1ce83c11c7d40fcd
Sep 16 14:47:16.099982 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:16.099962    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:47:16.678439 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:16.678387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74mws/must-gather-mfpfv" event={"ID":"82af91d8-9757-4089-abd5-228e7189a555","Type":"ContainerStarted","Data":"0c2f0a938a570e3818192251b74eb5c6942a707a4d20d6dd1ce83c11c7d40fcd"}
Sep 16 14:47:17.164757 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:17.164655    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log" limit=1024
Sep 16 14:47:17.687578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:17.687492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74mws/must-gather-mfpfv" event={"ID":"82af91d8-9757-4089-abd5-228e7189a555","Type":"ContainerStarted","Data":"8d973619d9fcab230c1ad1f26ba6eced3fb42ae77da613ecdb3367e85879627d"}
Sep 16 14:47:17.687578 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:17.687550    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74mws/must-gather-mfpfv" event={"ID":"82af91d8-9757-4089-abd5-228e7189a555","Type":"ContainerStarted","Data":"61f5c199686c2486333614ffe912edcbe4e6ab67eb0c53268def1ea69ad1b994"}
Sep 16 14:47:17.704431 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:17.704388    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-74mws/must-gather-mfpfv" podStartSLOduration=1.88480615 podStartE2EDuration="2.704374089s" totalImagesPullingTime="819.567939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:47:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:47:17.70298802 +0000 UTC m=+2568.190998096" watchObservedRunningTime="2026-09-16 14:47:17.704374089 +0000 UTC m=+2568.192384197"
Sep 16 14:47:18.481330 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:18.481294    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-px6lq_dd12c3f1-620d-4992-bb8b-68c1293ab5ab/global-pull-secret-syncer/0.log"
Sep 16 14:47:18.591086 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:18.591045    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5jpt6_0e8f9027-f385-4e8d-a1dd-f4829d480d77/konnectivity-agent/0.log"
Sep 16 14:47:18.699444 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:18.699408    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-34.ec2.internal_2bfb490af20cdfaeeb62e257a12524fb/haproxy/0.log"
Sep 16 14:47:23.461501 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:23.461460    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9jsjk_57f4ef0e-c570-4775-b965-1013161e7387/kuadrant-console-plugin/0.log"
Sep 16 14:47:23.531267 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:23.531239    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-crfbz_fe9b18fc-ba30-48ce-b927-953c794ea102/limitador/0.log"
Sep 16 14:47:24.951251 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:24.951216    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_725f126e-01b0-4981-8c71-8a3311ec2f97/alertmanager/0.log"
Sep 16 14:47:24.971734 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:24.971689    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_725f126e-01b0-4981-8c71-8a3311ec2f97/config-reloader/0.log"
Sep 16 14:47:24.993839 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:24.993806    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_725f126e-01b0-4981-8c71-8a3311ec2f97/kube-rbac-proxy-web/0.log"
Sep 16 14:47:25.013070 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.013040    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_725f126e-01b0-4981-8c71-8a3311ec2f97/kube-rbac-proxy/0.log"
Sep 16 14:47:25.033581 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.033558    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_725f126e-01b0-4981-8c71-8a3311ec2f97/kube-rbac-proxy-metric/0.log"
Sep 16 14:47:25.053648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.053624    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_725f126e-01b0-4981-8c71-8a3311ec2f97/prom-label-proxy/0.log"
Sep 16 14:47:25.075260 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.075231    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_725f126e-01b0-4981-8c71-8a3311ec2f97/init-config-reloader/0.log"
Sep 16 14:47:25.208208 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.207518    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8494cd7db7-nl5x9_6223c5b5-f143-4330-8e2c-303d614e486b/metrics-server/0.log"
Sep 16 14:47:25.333500 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.333467    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p6cgd_a2ca0e60-1d2a-4845-a983-f6831571be9d/node-exporter/0.log"
Sep 16 14:47:25.353951 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.353920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p6cgd_a2ca0e60-1d2a-4845-a983-f6831571be9d/kube-rbac-proxy/0.log"
Sep 16 14:47:25.371131 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.371106    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p6cgd_a2ca0e60-1d2a-4845-a983-f6831571be9d/init-textfile/0.log"
Sep 16 14:47:25.481863 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.481832    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6kgzp_1b190ccf-d681-4fd3-8e22-da2f1aafbea9/kube-rbac-proxy-main/0.log"
Sep 16 14:47:25.507397 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.507367    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6kgzp_1b190ccf-d681-4fd3-8e22-da2f1aafbea9/kube-rbac-proxy-self/0.log"
Sep 16 14:47:25.530953 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.530918    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6kgzp_1b190ccf-d681-4fd3-8e22-da2f1aafbea9/openshift-state-metrics/0.log"
Sep 16 14:47:25.590158 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.589779    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a1c6f5b-9412-4aad-9442-3585f02aaf0d/prometheus/0.log"
Sep 16 14:47:25.617802 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.617770    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a1c6f5b-9412-4aad-9442-3585f02aaf0d/config-reloader/0.log"
Sep 16 14:47:25.661158 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.660062    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a1c6f5b-9412-4aad-9442-3585f02aaf0d/thanos-sidecar/0.log"
Sep 16 14:47:25.695018 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.694936    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a1c6f5b-9412-4aad-9442-3585f02aaf0d/kube-rbac-proxy-web/0.log"
Sep 16 14:47:25.721200 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.721165    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a1c6f5b-9412-4aad-9442-3585f02aaf0d/kube-rbac-proxy/0.log"
Sep 16 14:47:25.759868 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.759793    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a1c6f5b-9412-4aad-9442-3585f02aaf0d/kube-rbac-proxy-thanos/0.log"
Sep 16 14:47:25.826245 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.826155    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a1c6f5b-9412-4aad-9442-3585f02aaf0d/init-config-reloader/0.log"
Sep 16 14:47:25.869345 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.869313    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-6q9pq_8d2725af-2346-43a9-b973-eff34547355a/prometheus-operator/0.log"
Sep 16 14:47:25.900970 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.900933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-6q9pq_8d2725af-2346-43a9-b973-eff34547355a/kube-rbac-proxy/0.log"
Sep 16 14:47:25.955123 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:25.955007    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-7lr9m_280b5349-c479-4d5d-96f2-de6262f6c471/prometheus-operator-admission-webhook/0.log"
Sep 16 14:47:26.093246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:26.092953    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8646694c88-ws45x_5424c0fc-020b-4532-8de4-361ef10b8771/thanos-query/0.log"
Sep 16 14:47:26.117402 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:26.117377    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8646694c88-ws45x_5424c0fc-020b-4532-8de4-361ef10b8771/kube-rbac-proxy-web/0.log"
Sep 16 14:47:26.142510 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:26.142478    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8646694c88-ws45x_5424c0fc-020b-4532-8de4-361ef10b8771/kube-rbac-proxy/0.log"
Sep 16 14:47:26.171808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:26.170328    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8646694c88-ws45x_5424c0fc-020b-4532-8de4-361ef10b8771/prom-label-proxy/0.log"
Sep 16 14:47:26.197295 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:26.197210    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8646694c88-ws45x_5424c0fc-020b-4532-8de4-361ef10b8771/kube-rbac-proxy-rules/0.log"
Sep 16 14:47:26.230705 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:26.230661    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8646694c88-ws45x_5424c0fc-020b-4532-8de4-361ef10b8771/kube-rbac-proxy-metrics/0.log"
Sep 16 14:47:28.715021 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.714978    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"]
Sep 16 14:47:28.720894 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.720870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.731808 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.731769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"]
Sep 16 14:47:28.846595 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.846553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l2fc\" (UniqueName: \"kubernetes.io/projected/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-kube-api-access-9l2fc\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.846791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.846625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-podres\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.846791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.846678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-lib-modules\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.846791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.846725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-proc\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.846791 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.846749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-sys\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.947781 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.947732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l2fc\" (UniqueName: \"kubernetes.io/projected/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-kube-api-access-9l2fc\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.947974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.947812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-podres\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.947974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.947869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-lib-modules\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.947974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.947916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-proc\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.947974 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.947941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-sys\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.948250 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.948123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-sys\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.948630 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.948597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-podres\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.948718 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.948694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-proc\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.948779 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.948716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-lib-modules\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.957247 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.957216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l2fc\" (UniqueName: \"kubernetes.io/projected/0b7614bc-05b8-45f1-ac01-f55dc79c4f12-kube-api-access-9l2fc\") pod \"perf-node-gather-daemonset-drsxn\" (UID: \"0b7614bc-05b8-45f1-ac01-f55dc79c4f12\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:28.975430 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:28.975355    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9b67fddbb-khhkb_f1c89cb1-ee03-4ebb-b1d3-bd7bfa2c835b/console/0.log"
Sep 16 14:47:29.014838 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:29.014809    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-b2677_47c5ce39-c04d-4810-ad73-63c0d59db14a/download-server/0.log"
Sep 16 14:47:29.046826 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:29.046791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:29.348215 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:29.347965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"]
Sep 16 14:47:29.350794 ip-10-0-137-34 kubenswrapper[2580]: W0916 14:47:29.350763    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0b7614bc_05b8_45f1_ac01_f55dc79c4f12.slice/crio-d7c23c044670c2f1a477c57a95a94bd014a06835e951875a3443bd27b9c76695 WatchSource:0}: Error finding container d7c23c044670c2f1a477c57a95a94bd014a06835e951875a3443bd27b9c76695: Status 404 returned error can't find the container with id d7c23c044670c2f1a477c57a95a94bd014a06835e951875a3443bd27b9c76695
Sep 16 14:47:29.798993 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:29.798949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn" event={"ID":"0b7614bc-05b8-45f1-ac01-f55dc79c4f12","Type":"ContainerStarted","Data":"16d02ebbab506c05d93a1eeab5a3d5d96c28e908a71666ab26b1cb4b76c55688"}
Sep 16 14:47:29.799491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:29.799022    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:29.799491 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:29.799059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn" event={"ID":"0b7614bc-05b8-45f1-ac01-f55dc79c4f12","Type":"ContainerStarted","Data":"d7c23c044670c2f1a477c57a95a94bd014a06835e951875a3443bd27b9c76695"}
Sep 16 14:47:29.817470 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:29.817411    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn" podStartSLOduration=1.817396294 podStartE2EDuration="1.817396294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:47:29.816097725 +0000 UTC m=+2580.304107803" watchObservedRunningTime="2026-09-16 14:47:29.817396294 +0000 UTC m=+2580.305406371"
Sep 16 14:47:30.887844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:30.887813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f9rpw_7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0/dns/0.log"
Sep 16 14:47:30.909607 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:30.909576    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f9rpw_7f0e0b1e-3871-47b7-8ddb-1b2716dcc0e0/kube-rbac-proxy/0.log"
Sep 16 14:47:30.984181 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:30.984131    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pv8ks_d72c7f4b-fbd4-496b-b66c-800629e3a75e/dns-node-resolver/0.log"
Sep 16 14:47:31.556356 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:31.556325    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-75b95bb9b8-mprfv_5d913ed1-3adb-4f64-96a2-d70d06616ae1/registry/0.log"
Sep 16 14:47:31.581686 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:31.581644    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mns9d_a01415cc-8ef7-4e60-8deb-0b314dd4fe3d/node-ca/0.log"
Sep 16 14:47:33.304220 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:33.304185    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b574975b8-5f7w7_483a6573-d5b6-4526-9406-878ea7adeebb/kube-auth-proxy/0.log"
Sep 16 14:47:33.377650 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:33.377626    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-wfbl9_c71e9728-1ad3-4657-a332-91d21e89b914/payload-pre-processing/0.log"
Sep 16 14:47:33.400681 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:33.400645    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-28hd7_e27dd0dd-1f25-4d76-84de-32602bba121c/payload-processing/0.log"
Sep 16 14:47:34.266650 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:34.266616    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ghpvl_95b03975-d212-411b-92dd-db3740693f1f/serve-healthcheck-canary/0.log"
Sep 16 14:47:34.849698 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:34.849667    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fccsf_eba65688-a180-4429-8372-fd32ccd5532d/kube-rbac-proxy/0.log"
Sep 16 14:47:34.869886 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:34.869860    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fccsf_eba65688-a180-4429-8372-fd32ccd5532d/exporter/0.log"
Sep 16 14:47:34.890687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:34.890646    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fccsf_eba65688-a180-4429-8372-fd32ccd5532d/extractor/0.log"
Sep 16 14:47:35.817157 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:35.817113    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-drsxn"
Sep 16 14:47:37.085732 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:37.085703    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f4fb96fc-phgkb_f5ce0a0c-9cdc-4ccf-bf82-82f18536f6ea/manager/0.log"
Sep 16 14:47:37.687987 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:37.687956    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6d9877-vt696_c3e88296-ead3-46fe-88d2-0f6f9a38b640/manager/0.log"
Sep 16 14:47:37.706160 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:37.706124    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6d9877-vt696_c3e88296-ead3-46fe-88d2-0f6f9a38b640/copy-manifests/0.log"
Sep 16 14:47:39.125938 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:39.125850    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-654b6f4fd4-vd82q_728deb49-1351-4c2c-bb93-aa0084090a73/manager/0.log"
Sep 16 14:47:39.395963 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:39.395881    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b574975b8-5f7w7_483a6573-d5b6-4526-9406-878ea7adeebb/kube-auth-proxy/0.log"
Sep 16 14:47:39.442739 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:39.442705    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-wfbl9_c71e9728-1ad3-4657-a332-91d21e89b914/payload-pre-processing/0.log"
Sep 16 14:47:39.456648 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:39.456614    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-28hd7_e27dd0dd-1f25-4d76-84de-32602bba121c/payload-processing/0.log"
Sep 16 14:47:41.638981 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:41.638943    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b574975b8-5f7w7_483a6573-d5b6-4526-9406-878ea7adeebb/kube-auth-proxy/0.log"
Sep 16 14:47:41.682845 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:41.682816    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-wfbl9_c71e9728-1ad3-4657-a332-91d21e89b914/payload-pre-processing/0.log"
Sep 16 14:47:41.695560 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:41.695535    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-28hd7_e27dd0dd-1f25-4d76-84de-32602bba121c/payload-processing/0.log"
Sep 16 14:47:44.200687 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:44.200652    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-xqbm5_1b511e3c-f247-4dbd-bfbe-c8256bcae36c/migrator/0.log"
Sep 16 14:47:44.218961 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:44.218934    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-xqbm5_1b511e3c-f247-4dbd-bfbe-c8256bcae36c/graceful-termination/0.log"
Sep 16 14:47:45.727246 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:45.727220    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fx94s_206c7143-17a7-46b9-93b9-c1509d26ccc1/kube-multus-additional-cni-plugins/0.log"
Sep 16 14:47:45.745175 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:45.745129    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fx94s_206c7143-17a7-46b9-93b9-c1509d26ccc1/egress-router-binary-copy/0.log"
Sep 16 14:47:45.763015 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:45.762989    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fx94s_206c7143-17a7-46b9-93b9-c1509d26ccc1/cni-plugins/0.log"
Sep 16 14:47:45.782270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:45.782251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fx94s_206c7143-17a7-46b9-93b9-c1509d26ccc1/bond-cni-plugin/0.log"
Sep 16 14:47:45.800270 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:45.800249    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fx94s_206c7143-17a7-46b9-93b9-c1509d26ccc1/routeoverride-cni/0.log"
Sep 16 14:47:45.818418 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:45.818363    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fx94s_206c7143-17a7-46b9-93b9-c1509d26ccc1/whereabouts-cni-bincopy/0.log"
Sep 16 14:47:45.837445 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:45.837426    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fx94s_206c7143-17a7-46b9-93b9-c1509d26ccc1/whereabouts-cni/0.log"
Sep 16 14:47:46.084340 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:46.084247    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-txpr4_984c2616-a7ee-4b42-ae38-bac0e2aa1e5a/kube-multus/0.log"
Sep 16 14:47:46.110844 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:46.110814    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pq29s_24c0e5cb-4a9e-4377-8e92-61022a699d47/network-metrics-daemon/0.log"
Sep 16 14:47:46.128414 ip-10-0-137-34 kubenswrapper[2580]: I0916 14:47:46.128381    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pq29s_24c0e5cb-4a9e-4377-8e92-61022a699d47/kube-rbac-proxy/0.log"