Sep 10 15:13:52.414811 ip-10-0-141-193 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 15:13:52.414826 ip-10-0-141-193 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 15:13:52.414836 ip-10-0-141-193 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 15:13:52.415076 ip-10-0-141-193 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 15:14:02.644542 ip-10-0-141-193 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 15:14:02.644559 ip-10-0-141-193 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 77b6d51732e94dca9ba59419cbc70e0e --
Sep 10 15:15:59.358761 ip-10-0-141-193 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 15:15:59.730582 ip-10-0-141-193 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:15:59.730582 ip-10-0-141-193 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 15:15:59.730582 ip-10-0-141-193 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:15:59.730582 ip-10-0-141-193 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 15:15:59.730582 ip-10-0-141-193 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:15:59.732792 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.732646    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 15:15:59.737043 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737019    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:15:59.737043 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737037    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:15:59.737043 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737041    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:15:59.737043 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737044    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:15:59.737043 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737047    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:15:59.737043 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737051    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737054    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737057    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737060    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737062    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737065    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737067    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737070    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737073    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737075    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737078    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737081    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737084    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737086    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737089    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737091    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737094    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737099    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737102    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:15:59.737260 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737105    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737108    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737112    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737114    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737117    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737121    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737123    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737126    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737129    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737132    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737134    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737137    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737140    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737143    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737145    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737149    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737151    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737154    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737157    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737159    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:15:59.737728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737163    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737166    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737170    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737174    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737176    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737179    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737181    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737185    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737187    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737190    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737192    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737195    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737198    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737200    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737203    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737205    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737208    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737212    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737214    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:15:59.738219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737217    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737219    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737222    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737224    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737227    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737230    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737232    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737235    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737239    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737242    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737244    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737247    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737250    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737252    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737255    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737258    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737260    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737263    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737265    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737268    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:15:59.738715 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737275    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737278    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737280    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737284    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737288    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737291    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737293    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737296    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737298    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737301    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737303    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737306    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737309    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737311    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737314    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.737316    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738044    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738052    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738055    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738058    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:15:59.739194 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738062    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738065    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738068    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738071    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738075    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738077    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738080    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738082    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738085    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738088    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738093    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738096    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738099    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738103    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738106    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738109    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738111    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738114    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738117    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738120    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:15:59.739734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738122    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738125    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738128    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738130    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738133    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738136    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738138    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738141    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738143    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738146    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738148    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738151    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738154    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738156    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738159    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738163    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738165    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738168    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738170    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738173    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:15:59.740224 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738175    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738178    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738180    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738182    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738185    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738187    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738191    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738193    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738196    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738198    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738201    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738203    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738206    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738208    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738211    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738215    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738218    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738222    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738225    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:15:59.740732 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738228    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738230    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738233    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738235    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738238    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738241    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738244    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738247    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738249    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738252    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738254    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738257    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738259    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738261    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738264    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738266    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738269    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738271    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738274    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:15:59.741197 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738279    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738285    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738288    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738290    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738293    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738295    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738298    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738301    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738303    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738306    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738308    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738310    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738313    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738315    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738318    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738320    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738323    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738734    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738740    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:15:59.741680 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738744    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738748    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738752    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738755    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738758    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738760    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738764    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738767    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738770    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738773    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738776    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738778    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738781    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738783    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738788    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738791    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738794    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738797    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738799    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:15:59.742143 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738802    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738804    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738807    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738809    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738812    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738814    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738817    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738819    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738822    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738824    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738827    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738830    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738833    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738836    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738839    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738841    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738844    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738847    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738849    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738851    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:15:59.742733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738854    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738857    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738859    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738862    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738864    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738867    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738869    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738873    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738876    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738879    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738882    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738884    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738887    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738889    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738892    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738894    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738896    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738899    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738902    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738904    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:15:59.743233 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738907    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738909    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738913    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738916    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738921    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738924    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738926    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738929    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738932    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738934    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738937    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738939    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738942    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738944    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738947    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738949    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738952    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738954    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738957    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738961    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:15:59.743821 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738964    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738967    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738969    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738972    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738974    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738976    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738979    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738981    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738984    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738986    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738989    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738992    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738994    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738996    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.738999    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739002    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739004    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739007    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739076    2562 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739119    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739128    2562 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 15:15:59.744336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739132    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739136    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739140    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739144    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739148    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739151    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739154    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739157    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739161    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739164    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739169    2562 flags.go:64] FLAG: --cgroup-root=""
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739172    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739175    2562 flags.go:64] FLAG: --client-ca-file=""
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739178    2562 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739181    2562 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739185    2562 flags.go:64] FLAG: --cluster-domain=""
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739188    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739191    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739194    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739198    2562 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739202    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739205    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739209    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 15:15:59.744859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739212    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739215    2562 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739218    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739221    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739225    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739228    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739232    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739235    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739238    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739240    2562 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739243    2562 flags.go:64] FLAG: --enable-server="true"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739246    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739254    2562 flags.go:64] FLAG: --event-burst="100"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739257    2562 flags.go:64] FLAG: --event-qps="50"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739260    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739263    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739266    2562 flags.go:64] FLAG: --eviction-hard=""
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739270    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739273    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739276    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739279    2562 flags.go:64] FLAG: --eviction-soft=""
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739282    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739285    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739288    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739291    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 15:15:59.745422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739294    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739297    2562 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739300    2562 flags.go:64] FLAG: --feature-gates=""
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739304    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739307    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739310    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739313    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739316    2562 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739319    2562 flags.go:64] FLAG: --help="false"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739322    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-141-193.ec2.internal"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739325    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739329    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739331    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739335    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739338    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739341    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739343    2562 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739346    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739348    2562 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739351    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739354    2562 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739357    2562 flags.go:64] FLAG: --kube-reserved=""
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739360    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739363    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 15:15:59.746058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739366    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739368    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739372    2562 flags.go:64] FLAG: --lock-file=""
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739375    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739378    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739381    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739386    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739392    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739395    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739397    2562 flags.go:64] FLAG: --logging-format="text"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739400    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739403    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739406    2562 flags.go:64] FLAG: --manifest-url=""
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739409    2562 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739414    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739417    2562 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739420    2562 flags.go:64] FLAG: --max-pods="110"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739423    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739426    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739432    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739460    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739464    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739468    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739471    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739479    2562 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 15:15:59.746717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739482    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739485    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739488    2562 flags.go:64] FLAG: --pod-cidr=""
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739491    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739497    2562 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739500    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739503    2562 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739506    2562 flags.go:64] FLAG: --port="10250"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739509    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739512    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a8ee4ef2a7f00cfc"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739515    2562 flags.go:64] FLAG: --qos-reserved=""
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739518    2562 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739521    2562 flags.go:64] FLAG: --register-node="true"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739524    2562 flags.go:64] FLAG: --register-with-taints=""
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739527    2562 flags.go:64] FLAG: --registry-burst="10"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739531    2562 flags.go:64] FLAG: --registry-qps="5"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739534    2562 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739536    2562 flags.go:64] FLAG: --reserved-memory=""
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739544    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739547    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739550    2562 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739553    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739556    2562 flags.go:64] FLAG: --runonce="false"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739559    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739561    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 15:15:59.747338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739564    2562 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739569    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739571    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739574    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739577    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739580    2562 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739583    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739586    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739589    2562 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739591    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739594    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739597    2562 flags.go:64] FLAG: --system-cgroups=""
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739600    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739605    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739607    2562 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739610    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739616    2562 flags.go:64] FLAG: --tls-min-version=""
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739619    2562 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739622    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739625    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739628    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739631    2562 flags.go:64] FLAG: --v="2"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739634    2562 flags.go:64] FLAG: --version="false"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739639    2562 flags.go:64] FLAG: --vmodule=""
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739642    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 15:15:59.747996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.739646    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739755    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739759    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739762    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739764    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739767    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739769    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739772    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739776    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739778    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739781    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739784    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739787    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739789    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739792    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739795    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739797    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739800    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739802    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739806    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:15:59.748606 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739808    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739811    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739814    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739816    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739820    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739822    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739825    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739829    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739833    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739835    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739838    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739846    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739849    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739853    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739856    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739860    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739864    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739867    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739870    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739873    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:15:59.749081 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739876    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739879    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739882    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739884    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739887    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739889    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739892    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739894    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739897    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739900    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739915    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739918    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739922    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739925    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739928    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739931    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739935    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739938    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739941    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739943    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:15:59.749628 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739946    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739948    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739951    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739954    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739960    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739963    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739965    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739968    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739970    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739973    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739975    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739978    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739982    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739984    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739987    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739991    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739994    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739997    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.739999    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:15:59.750112 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740002    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740005    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740009    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740012    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740015    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740018    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740021    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740024    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740028    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740032    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740035    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740038    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740040    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740043    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740045    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740048    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740050    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740053    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740056    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:15:59.750595 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740059    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.740061    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.740584    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.747616    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.747632    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.748297    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.748304    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748367    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748372    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748375    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748379    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748382    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748385    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748388    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748390    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:15:59.751058 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748393    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748396    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748398    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748401    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748404    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748406    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748409    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748412    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748415    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748418    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748420    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748423    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748425    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748428    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748430    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748433    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748435    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748438    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748441    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748443    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748446    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:15:59.751486 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748473    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748476    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748478    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748481    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748484    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748487    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748489    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748492    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748495    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748497    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748500    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748502    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748505    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748508    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748510    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748513    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748515    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748517    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748520    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748523    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:15:59.752055 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748525    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748528    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748530    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748533    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748537    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748540    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748543    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748547    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748551    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748554    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748556    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748559    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748562    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748570    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748573    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748575    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748578    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748581    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748583    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:15:59.752556 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748586    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748588    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748591    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748594    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748596    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748598    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748601    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748604    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748606    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748609    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748612    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748614    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748617    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748620    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748623    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748625    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748628    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748630    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748632    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748635    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:15:59.753017 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748637    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748640    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748642    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748645    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748647    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748650    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748654    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748663    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748666    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748668    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748671    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.748676    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748821    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748826    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:15:59.753518 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748829    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748832    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748835    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748838    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748841    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748844    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748849    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748851    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748854    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748857    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748861    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748864    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748867    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748870    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748873    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748875    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748878    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748880    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748883    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:15:59.753864 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748885    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748888    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748890    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748893    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748896    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748898    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748901    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748909    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748912    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748914    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748917    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748919    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748922    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748924    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748927    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748929    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748932    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748936    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748939    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748942    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:15:59.754315 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748944    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748947    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748950    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748952    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748955    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748957    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748960    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748963    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748965    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748967    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748970    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748972    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748975    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748977    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748979    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748982    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748985    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748987    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748990    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.748992    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:15:59.754851 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749000    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749003    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749005    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749008    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749010    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749013    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749015    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749018    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749020    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749024    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749027    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749030    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749032    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749035    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749037    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749039    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749042    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749045    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749047    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749050    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:15:59.755340 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749052    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749055    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749057    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749060    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749062    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749065    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749067    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749071    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749073    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749076    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749078    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749080    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749083    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749091    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749094    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749097    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749099    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:15:59.755911 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:15:59.749102    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:15:59.756318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.749107    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:15:59.756318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.749229    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 15:15:59.756318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.751527    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 15:15:59.756318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.752327    2562 server.go:1019] "Starting client certificate rotation"
Sep 10 15:15:59.756318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.752431    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 15:15:59.756318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.752481    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 15:15:59.756318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.754802    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 15:15:59.760428 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.760405    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 15:15:59.769493 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.769475    2562 log.go:25] "Validated CRI v1 runtime API"
Sep 10 15:15:59.774302 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.774287    2562 log.go:25] "Validated CRI v1 image API"
Sep 10 15:15:59.775120 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.775108    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 15:15:59.777355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.777322    2562 fs.go:135] Filesystem UUIDs: map[1c5d781e-eff2-4611-91e9-f2b940c71ac4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f30b1589-7258-4c0b-94c6-b31d460f6d60:/dev/nvme0n1p3]
Sep 10 15:15:59.777433 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.777352    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 15:15:59.781562 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.781545    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 15:15:59.782822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.782714    2562 manager.go:216] Machine: {Timestamp:2026-09-10 15:15:59.781413373 +0000 UTC m=+0.317368927 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090143 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28f29471ad617429aa26dd7bf650db SystemUUID:ec28f294-71ad-6174-29aa-26dd7bf650db BootID:77b6d517-32e9-4dca-9ba5-9419cbc70e0e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:03:58:c9:1b:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:03:58:c9:1b:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:e4:37:8c:4f:be Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 15:15:59.782822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.782816    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 15:15:59.782930 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.782890    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 15:15:59.785800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.785779    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 15:15:59.785932 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.785802    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-193.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 15:15:59.787054 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.787044    2562 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 15:15:59.787089 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.787056    2562 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 15:15:59.787089 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.787064    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 15:15:59.787765 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.787753    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 15:15:59.789321 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.789311    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 15:15:59.792588 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.792575    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 15:15:59.792705 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.792695    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 15:15:59.792737 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.792729    2562 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 15:15:59.792768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.792738    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 15:15:59.792768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.792752    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 15:15:59.792768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.792760    2562 kubelet.go:398] "Adding apiserver pod source"
Sep 10 15:15:59.792852 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.792775    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 15:15:59.793267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.793249    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 15:15:59.793324 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.793289    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 15:15:59.795446 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.795430    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 15:15:59.797983 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.797968    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 15:15:59.798069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798005    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 15:15:59.798747 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798733    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 15:15:59.798813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798754    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 15:15:59.798813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798763    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 15:15:59.798813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798770    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 15:15:59.798813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798782    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 15:15:59.798813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798790    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 15:15:59.799004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798823    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 15:15:59.799004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798831    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 15:15:59.799004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798841    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 15:15:59.799004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798849    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 15:15:59.799004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798866    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 15:15:59.799004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.798874    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 15:15:59.799523 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.799512    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 15:15:59.799584 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.799525    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 15:15:59.802224 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.802200    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 15:15:59.802416 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.802400    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-193.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 15:15:59.802507 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.802483    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 15:15:59.803432 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.803414    2562 server.go:1268] "Started kubelet"
Sep 10 15:15:59.803516 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.803480    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 15:15:59.803575 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.803531    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 15:15:59.803642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.803627    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 15:15:59.804381 ip-10-0-141-193 systemd[1]: Started Kubernetes Kubelet.
Sep 10 15:15:59.808179 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.808092    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 15:15:59.810280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.810264    2562 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 15:15:59.811518 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.811481    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 15:15:59.812989 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.812974    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tpv8g"
Sep 10 15:15:59.813745 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.813723    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 15:15:59.814244 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.814224    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 15:15:59.814381 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.814362    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:15:59.814551 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.813570    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-193.ec2.internal.18d3fe8d4b4b8589  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-193.ec2.internal,UID:ip-10-0-141-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-193.ec2.internal,},FirstTimestamp:2026-09-10 15:15:59.803303305 +0000 UTC m=+0.339258857,LastTimestamp:2026-09-10 15:15:59.803303305 +0000 UTC m=+0.339258857,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-193.ec2.internal,}"
Sep 10 15:15:59.815000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.814984    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 15:15:59.815000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.814986    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 15:15:59.815137 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815010    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 15:15:59.815137 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815021    2562 factory.go:55] Registering systemd factory
Sep 10 15:15:59.815137 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815034    2562 factory.go:223] Registration of the systemd container factory successfully
Sep 10 15:15:59.815260 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815142    2562 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 15:15:59.815260 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815150    2562 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 15:15:59.815260 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815217    2562 factory.go:153] Registering CRI-O factory
Sep 10 15:15:59.815260 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815227    2562 factory.go:223] Registration of the crio container factory successfully
Sep 10 15:15:59.815440 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815276    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 15:15:59.815440 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815292    2562 factory.go:103] Registering Raw factory
Sep 10 15:15:59.815440 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.815306    2562 manager.go:1196] Started watching for new ooms in manager
Sep 10 15:15:59.816047 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.816033    2562 manager.go:349] Starting recovery of all containers
Sep 10 15:15:59.819375 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.819353    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tpv8g"
Sep 10 15:15:59.823398 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.823383    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:15:59.826538 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.826517    2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-193.ec2.internal\" not found" node="ip-10-0-141-193.ec2.internal"
Sep 10 15:15:59.828744 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.828706    2562 manager.go:354] Recovery completed
Sep 10 15:15:59.833373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.833351    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:15:59.835391 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.835373    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:15:59.835524 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.835408    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:15:59.835524 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.835423    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:15:59.836028 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.836013    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 15:15:59.836028 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.836027    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 15:15:59.836140 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.836028    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:15:59.836140 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.836045    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 15:15:59.836755 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.836741    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:15:59.836821 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.836767    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:15:59.836821 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.836780    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:15:59.837313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.837300    2562 policy_none.go:49] "None policy: Start"
Sep 10 15:15:59.837366 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.837319    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 15:15:59.837366 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.837332    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 15:15:59.838555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.838541    2562 policy_none.go:47] "Start"
Sep 10 15:15:59.880160 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.880144    2562 manager.go:337] "Starting Device Plugin manager"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.880185    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.880195    2562 server.go:87] "Starting device plugin registration server"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.880430    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.880442    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.880558    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.880635    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.880643    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.881129    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 15:15:59.893445 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.881171    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:15:59.947210 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.947185    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 15:15:59.948343 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.948327    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 15:15:59.948430 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.948380    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 15:15:59.948430 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.948406    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 15:15:59.948546 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.948512    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 15:15:59.951021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.950986    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:15:59.981465 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.981396    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:15:59.982336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.982317    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:15:59.982397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.982356    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:15:59.982397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.982372    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:15:59.982397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.982393    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-193.ec2.internal"
Sep 10 15:15:59.991202 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:15:59.991042    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-193.ec2.internal"
Sep 10 15:15:59.991301 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:15:59.991211    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-193.ec2.internal\": node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.007574 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.007550    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.048580 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.048552    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal"]
Sep 10 15:16:00.048670 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.048621    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:16:00.050195 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.050179    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:16:00.050282 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.050213    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:16:00.050282 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.050229    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:16:00.052514 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.052499    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:16:00.052615 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.052600    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.052662 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.052642    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:16:00.053225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.053204    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:16:00.053225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.053217    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:16:00.053337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.053230    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:16:00.053337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.053238    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:16:00.053337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.053250    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:16:00.053337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.053240    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:16:00.055379 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.055367    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.055423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.055394    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:16:00.056114 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.056098    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:16:00.056189 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.056130    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:16:00.056189 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.056146    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:16:00.078977 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.078957    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-193.ec2.internal\" not found" node="ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.083085 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.083070    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-193.ec2.internal\" not found" node="ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.107711 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.107693    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.117032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.117014    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/330ea1231cf8707c9833237d01f0cf58-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal\" (UID: \"330ea1231cf8707c9833237d01f0cf58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.117127 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.117110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/756919c48f155444a9a2e4c796c6a07b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-193.ec2.internal\" (UID: \"756919c48f155444a9a2e4c796c6a07b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.117162 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.117137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/330ea1231cf8707c9833237d01f0cf58-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal\" (UID: \"330ea1231cf8707c9833237d01f0cf58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.208776 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.208744    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.218161 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.218133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/330ea1231cf8707c9833237d01f0cf58-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal\" (UID: \"330ea1231cf8707c9833237d01f0cf58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.218251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.218169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/330ea1231cf8707c9833237d01f0cf58-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal\" (UID: \"330ea1231cf8707c9833237d01f0cf58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.218251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.218205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/756919c48f155444a9a2e4c796c6a07b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-193.ec2.internal\" (UID: \"756919c48f155444a9a2e4c796c6a07b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.218251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.218225    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/330ea1231cf8707c9833237d01f0cf58-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal\" (UID: \"330ea1231cf8707c9833237d01f0cf58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.218364 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.218252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/756919c48f155444a9a2e4c796c6a07b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-193.ec2.internal\" (UID: \"756919c48f155444a9a2e4c796c6a07b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.218364 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.218293    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/330ea1231cf8707c9833237d01f0cf58-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal\" (UID: \"330ea1231cf8707c9833237d01f0cf58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.309645 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.309558    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.382079 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.382048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.387595 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.387271    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:00.410001 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.409978    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.510095 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.510064    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.615728 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.615630    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.716045 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.716002    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.752304 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.752270    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 15:16:00.753029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.752412    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 15:16:00.753029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.752434    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 15:16:00.773981 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.773957    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:16:00.816951 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.816919    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:00.820901 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.820871    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 15:11:00 +0000 UTC" deadline="2028-06-07 18:47:13.784695326 +0000 UTC"
Sep 10 15:16:00.820901 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:00.820901    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15267h31m12.963796728s"
Sep 10 15:16:00.917589 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:00.917558    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:01.017963 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.017924    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-193.ec2.internal\" not found"
Sep 10 15:16:01.060208 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:01.060173    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756919c48f155444a9a2e4c796c6a07b.slice/crio-71009474ed9e2dd4f38d59b9d36c9ff1b42bb145cea5d17fc60c6f248610580a WatchSource:0}: Error finding container 71009474ed9e2dd4f38d59b9d36c9ff1b42bb145cea5d17fc60c6f248610580a: Status 404 returned error can't find the container with id 71009474ed9e2dd4f38d59b9d36c9ff1b42bb145cea5d17fc60c6f248610580a
Sep 10 15:16:01.060718 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:01.060694    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330ea1231cf8707c9833237d01f0cf58.slice/crio-e45a7b2d4abe711fd7de3266ddc64b064c189a1f0fb0a3ba070b565b897c18a0 WatchSource:0}: Error finding container e45a7b2d4abe711fd7de3266ddc64b064c189a1f0fb0a3ba070b565b897c18a0: Status 404 returned error can't find the container with id e45a7b2d4abe711fd7de3266ddc64b064c189a1f0fb0a3ba070b565b897c18a0
Sep 10 15:16:01.065197 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.065182    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:16:01.103373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.103337    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:16:01.114525 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.114494    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:01.127881 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.127857    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 15:16:01.128940 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.128927    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal"
Sep 10 15:16:01.139813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.139794    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 15:16:01.794914 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.794703    2562 apiserver.go:52] "Watching apiserver"
Sep 10 15:16:01.803650 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.803616    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 15:16:01.804172 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.804124    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-2htvg","openshift-multus/multus-additional-cni-plugins-qbn6b","openshift-multus/network-metrics-daemon-t8jf6","kube-system/global-pull-secret-syncer-z98zv","kube-system/konnectivity-agent-pqh9r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal","openshift-multus/multus-fzrqw","openshift-network-diagnostics/network-check-target-dhxd7","openshift-network-operator/iptables-alerter-52wx5","openshift-ovn-kubernetes/ovnkube-node-dn2vx","kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal","openshift-cluster-node-tuning-operator/tuned-vlhzg","openshift-dns/node-resolver-pwcph"]
Sep 10 15:16:01.807369 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.807059    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:01.807369 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.807147    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:01.809591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.809566    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.816032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.814140    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 15:16:01.816032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.814394    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 15:16:01.816032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.814640    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 15:16:01.816032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.814721    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-brv7s\""
Sep 10 15:16:01.816032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.814991    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 15:16:01.816032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.815059    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 15:16:01.816032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.815252    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 15:16:01.816987 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.816644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:01.816987 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.816748    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:01.819743 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.819726    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:01.822345 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.821925    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.822345 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.822199    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hf496\""
Sep 10 15:16:01.823571 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.823537    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 15:16:01.823918 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.823900    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 15:16:01.825235 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.825199    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 15:16:01.826013 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.825436    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:01.827339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.826532    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 15:16:01.827339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.826564    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 15:16:01.827339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.826581    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 15:16:01.827339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.826814    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 15:16:01.827339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.827068    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 15:16:01.827339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.827097    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pqmdk\""
Sep 10 15:16:01.827339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.827343    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.827908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.827937    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.827943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cnibin\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828012    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828043    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgpmm\" (UniqueName: \"kubernetes.io/projected/a454b8ce-8961-450d-ada9-bdb7489af486-kube-api-access-fgpmm\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828111    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-system-cni-dir\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828190    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87qmq\" (UniqueName: \"kubernetes.io/projected/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-kube-api-access-87qmq\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828294    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fq8tg\""
Sep 10 15:16:01.828463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828436    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 15:16:01.829178 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.828504    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 15:16:01.829729 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.829670    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.829890 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.829872    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:16:01.830591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.830218    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 15:16:01.830591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.830242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9pct9\""
Sep 10 15:16:01.830591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.830565    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 15:16:01.832298 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.832281    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 15:16:01.832424 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.832406    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k98s5\""
Sep 10 15:16:01.832633 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.832617    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.834833 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.834811    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:01.835679 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.835657    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 15:16:01.835892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.835873    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z9l6k\""
Sep 10 15:16:01.836113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.836094    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 15:16:01.836442 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.836418    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 15:16:01.836703 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.836685    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 15:16:01.837283 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.837265    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 15:16:01.837638 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.837618    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.837925 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.837906    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 15:16:01.838197 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.838177    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f4mcd\""
Sep 10 15:16:01.840078 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.840059    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:01.840220 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.840133    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:01.841437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.841416    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:16:01.841588 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.841425    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 15:16:01.841688 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.841670    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m2w42\""
Sep 10 15:16:01.849232 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.849207    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bjvj5"
Sep 10 15:16:01.859868 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.859842    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bjvj5"
Sep 10 15:16:01.921488 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.921439    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 15:16:01.922768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.922745    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:16:01.929055 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-sys-fs\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.929148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-lib-modules\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.929148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-systemd-units\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xczt\" (UniqueName: \"kubernetes.io/projected/7f76b831-7e7f-4a52-ba28-4336032a7d9c-kube-api-access-6xczt\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-hosts-file\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:01.929148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/62582b57-cacc-4c8f-8bb7-d2b2aaa19a20-konnectivity-ca\") pod \"konnectivity-agent-pqh9r\" (UID: \"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20\") " pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysctl-d\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-run\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-log-socket\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-cni-bin\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovnkube-config\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93274098-75c7-45d2-8aa8-6329b6973bfa-host-slash\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz4v5\" (UniqueName: \"kubernetes.io/projected/da2cfda9-d1cb-41e0-978c-8c14f669fe90-kube-api-access-sz4v5\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-device-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-etc-selinux\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysctl-conf\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-node-log\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovn-node-metrics-cert\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovnkube-script-lib\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-system-cni-dir\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.929460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-hostroot\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65dd5\" (UniqueName: \"kubernetes.io/projected/55e08010-50ce-42cf-96d0-a0421667d595-kube-api-access-65dd5\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929544    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-kubernetes\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-systemd\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-cni-netd\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929624    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87qmq\" (UniqueName: \"kubernetes.io/projected/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-kube-api-access-87qmq\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929658    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysconfig\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929710    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx94p\" (UniqueName: \"kubernetes.io/projected/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-kube-api-access-xx94p\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-netns\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-conf-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-multus-certs\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shpk2\" (UniqueName: \"kubernetes.io/projected/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-kube-api-access-shpk2\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-registration-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.930262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/62582b57-cacc-4c8f-8bb7-d2b2aaa19a20-agent-certs\") pod \"konnectivity-agent-pqh9r\" (UID: \"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20\") " pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929946    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-env-overrides\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.929998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93274098-75c7-45d2-8aa8-6329b6973bfa-iptables-alerter-script\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930020    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-k8s-cni-cncf-io\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-kubelet\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-etc-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-cni-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930127    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-cni-multus\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-etc-kubernetes\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-run-netns\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/55e08010-50ce-42cf-96d0-a0421667d595-multus-daemon-config\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-host\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930285    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-tuned\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55e08010-50ce-42cf-96d0-a0421667d595-cni-binary-copy\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-cni-bin\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930362    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-kubelet\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-var-lib-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvc6\" (UniqueName: \"kubernetes.io/projected/93274098-75c7-45d2-8aa8-6329b6973bfa-kube-api-access-7lvc6\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcwsh\" (UniqueName: \"kubernetes.io/projected/ee142568-3f98-42ca-bae6-66ad70df0590-kube-api-access-lcwsh\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cnibin\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75f17da5-a2cb-460c-b5ad-5804f13505dc-dbus\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930607    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-socket-dir-parent\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-tmp-dir\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da2cfda9-d1cb-41e0-978c-8c14f669fe90-host\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930717    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-sys\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-run-ovn-kubernetes\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-modprobe-d\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.931934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-tmp\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930835    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75f17da5-a2cb-460c-b5ad-5804f13505dc-kubelet-config\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930861    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-slash\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930886    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-ovn\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930912    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-system-cni-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da2cfda9-d1cb-41e0-978c-8c14f669fe90-serviceca\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-socket-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.930999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-systemd\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.931026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-var-lib-kubelet\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.931051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-cnibin\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.931080    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.931110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgpmm\" (UniqueName: \"kubernetes.io/projected/a454b8ce-8961-450d-ada9-bdb7489af486-kube-api-access-fgpmm\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.931670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-system-cni-dir\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.932021    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.932134    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs podName:a454b8ce-8961-450d-ada9-bdb7489af486 nodeName:}" failed. No retries permitted until 2026-09-10 15:16:02.432097793 +0000 UTC m=+2.968053359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs") pod "network-metrics-daemon-t8jf6" (UID: "a454b8ce-8961-450d-ada9-bdb7489af486") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:01.932735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.932358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cnibin\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.933565 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.932862    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.933628 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.933607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.934052 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.934027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.934249 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.934226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.940405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.939244    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 15:16:01.940405 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.940126    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:16:01.940405 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.940145    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:16:01.940405 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.940160    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wjggj for pod openshift-network-diagnostics/network-check-target-dhxd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:01.940405 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:01.940244    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj podName:8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:02.440225241 +0000 UTC m=+2.976180801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wjggj" (UniqueName: "kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj") pod "network-check-target-dhxd7" (UID: "8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:01.945366 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.945339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgpmm\" (UniqueName: \"kubernetes.io/projected/a454b8ce-8961-450d-ada9-bdb7489af486-kube-api-access-fgpmm\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:01.945480 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.945410    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87qmq\" (UniqueName: \"kubernetes.io/projected/51ffc72e-a4eb-4f7a-be2f-e974772b84aa-kube-api-access-87qmq\") pod \"multus-additional-cni-plugins-qbn6b\" (UID: \"51ffc72e-a4eb-4f7a-be2f-e974772b84aa\") " pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:01.956112 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.956062    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal" event={"ID":"330ea1231cf8707c9833237d01f0cf58","Type":"ContainerStarted","Data":"e45a7b2d4abe711fd7de3266ddc64b064c189a1f0fb0a3ba070b565b897c18a0"}
Sep 10 15:16:01.957446 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:01.957419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal" event={"ID":"756919c48f155444a9a2e4c796c6a07b","Type":"ContainerStarted","Data":"71009474ed9e2dd4f38d59b9d36c9ff1b42bb145cea5d17fc60c6f248610580a"}
Sep 10 15:16:02.031542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75f17da5-a2cb-460c-b5ad-5804f13505dc-dbus\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:02.031542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-socket-dir-parent\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.031542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-tmp-dir\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:02.031542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75f17da5-a2cb-460c-b5ad-5804f13505dc-dbus\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:02.031542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031509    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-socket-dir-parent\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.031542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da2cfda9-d1cb-41e0-978c-8c14f669fe90-host\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:02.031542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da2cfda9-d1cb-41e0-978c-8c14f669fe90-host\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-sys\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-run-ovn-kubernetes\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031632    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-modprobe-d\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031653    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-tmp\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75f17da5-a2cb-460c-b5ad-5804f13505dc-kubelet-config\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031687    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-sys\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-slash\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-ovn\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-tmp-dir\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-system-cni-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031777    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da2cfda9-d1cb-41e0-978c-8c14f669fe90-serviceca\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031816    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-modprobe-d\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031819    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-slash\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-socket-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-systemd\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-run-ovn-kubernetes\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031889    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-var-lib-kubelet\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-cnibin\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031944    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-sys-fs\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-lib-modules\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-systemd-units\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-socket-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xczt\" (UniqueName: \"kubernetes.io/projected/7f76b831-7e7f-4a52-ba28-4336032a7d9c-kube-api-access-6xczt\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-hosts-file\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/62582b57-cacc-4c8f-8bb7-d2b2aaa19a20-konnectivity-ca\") pod \"konnectivity-agent-pqh9r\" (UID: \"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20\") " pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysctl-d\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-run\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-log-socket\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-cni-bin\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovnkube-config\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93274098-75c7-45d2-8aa8-6329b6973bfa-host-slash\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.031917    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-ovn\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93274098-75c7-45d2-8aa8-6329b6973bfa-host-slash\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.032924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysctl-d\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-systemd\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032393    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-systemd-units\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032435    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-lib-modules\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75f17da5-a2cb-460c-b5ad-5804f13505dc-kubelet-config\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-system-cni-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032488    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-cni-bin\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4v5\" (UniqueName: \"kubernetes.io/projected/da2cfda9-d1cb-41e0-978c-8c14f669fe90-kube-api-access-sz4v5\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-cnibin\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-run\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032579    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-device-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032594    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-var-lib-kubelet\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032623    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-sys-fs\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032641    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-device-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-log-socket\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-etc-selinux\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-hosts-file\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysctl-conf\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.033856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-node-log\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-etc-selinux\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032744    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovn-node-metrics-cert\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovnkube-script-lib\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032800    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-hostroot\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65dd5\" (UniqueName: \"kubernetes.io/projected/55e08010-50ce-42cf-96d0-a0421667d595-kube-api-access-65dd5\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da2cfda9-d1cb-41e0-978c-8c14f669fe90-serviceca\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-kubernetes\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysctl-conf\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-systemd\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-cni-netd\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-hostroot\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysconfig\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx94p\" (UniqueName: \"kubernetes.io/projected/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-kube-api-access-xx94p\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.032993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-netns\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033028    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovnkube-config\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-conf-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.034794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-multus-certs\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-run-systemd\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033131    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shpk2\" (UniqueName: \"kubernetes.io/projected/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-kube-api-access-shpk2\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033148    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-cni-netd\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-registration-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033184    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-sysconfig\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/62582b57-cacc-4c8f-8bb7-d2b2aaa19a20-konnectivity-ca\") pod \"konnectivity-agent-pqh9r\" (UID: \"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20\") " pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/62582b57-cacc-4c8f-8bb7-d2b2aaa19a20-agent-certs\") pod \"konnectivity-agent-pqh9r\" (UID: \"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20\") " pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033083    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-conf-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-env-overrides\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033262    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-multus-certs\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93274098-75c7-45d2-8aa8-6329b6973bfa-iptables-alerter-script\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033315    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-k8s-cni-cncf-io\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033342    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-kubelet\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033346    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovnkube-script-lib\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033368    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-etc-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.033382    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:02.035720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-netns\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-cni-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.033431    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret podName:75f17da5-a2cb-460c-b5ad-5804f13505dc nodeName:}" failed. No retries permitted until 2026-09-10 15:16:02.533418911 +0000 UTC m=+3.069374452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret") pod "global-pull-secret-syncer-z98zv" (UID: "75f17da5-a2cb-460c-b5ad-5804f13505dc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-kubernetes\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-cni-multus\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-multus-cni-dir\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-cni-multus\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033522    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-etc-kubernetes\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033550    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-registration-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-run-netns\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/55e08010-50ce-42cf-96d0-a0421667d595-multus-daemon-config\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-node-log\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-host\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-run-k8s-cni-cncf-io\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-tuned\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55e08010-50ce-42cf-96d0-a0421667d595-cni-binary-copy\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.036664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033761    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-cni-bin\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-kubelet\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-var-lib-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvc6\" (UniqueName: \"kubernetes.io/projected/93274098-75c7-45d2-8aa8-6329b6973bfa-kube-api-access-7lvc6\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f76b831-7e7f-4a52-ba28-4336032a7d9c-env-overrides\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwsh\" (UniqueName: \"kubernetes.io/projected/ee142568-3f98-42ca-bae6-66ad70df0590-kube-api-access-lcwsh\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.033999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-cni-bin\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-run-netns\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-host-var-lib-kubelet\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-var-lib-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034198    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55e08010-50ce-42cf-96d0-a0421667d595-etc-kubernetes\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034342    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93274098-75c7-45d2-8aa8-6329b6973bfa-iptables-alerter-script\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-kubelet\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-etc-openvswitch\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034531    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-host\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ee142568-3f98-42ca-bae6-66ad70df0590-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f76b831-7e7f-4a52-ba28-4336032a7d9c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.037543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034676    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/55e08010-50ce-42cf-96d0-a0421667d595-multus-daemon-config\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.038421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.034826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55e08010-50ce-42cf-96d0-a0421667d595-cni-binary-copy\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.038421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.035639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-tmp\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.038421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.036415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f76b831-7e7f-4a52-ba28-4336032a7d9c-ovn-node-metrics-cert\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.038421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.036760    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-etc-tuned\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.038421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.037636    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/62582b57-cacc-4c8f-8bb7-d2b2aaa19a20-agent-certs\") pod \"konnectivity-agent-pqh9r\" (UID: \"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20\") " pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:02.042396 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.042360    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xczt\" (UniqueName: \"kubernetes.io/projected/7f76b831-7e7f-4a52-ba28-4336032a7d9c-kube-api-access-6xczt\") pod \"ovnkube-node-dn2vx\" (UID: \"7f76b831-7e7f-4a52-ba28-4336032a7d9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.042803 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.042778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz4v5\" (UniqueName: \"kubernetes.io/projected/da2cfda9-d1cb-41e0-978c-8c14f669fe90-kube-api-access-sz4v5\") pod \"node-ca-2htvg\" (UID: \"da2cfda9-d1cb-41e0-978c-8c14f669fe90\") " pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:02.044788 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.044753    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65dd5\" (UniqueName: \"kubernetes.io/projected/55e08010-50ce-42cf-96d0-a0421667d595-kube-api-access-65dd5\") pod \"multus-fzrqw\" (UID: \"55e08010-50ce-42cf-96d0-a0421667d595\") " pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.045012 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.044958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvc6\" (UniqueName: \"kubernetes.io/projected/93274098-75c7-45d2-8aa8-6329b6973bfa-kube-api-access-7lvc6\") pod \"iptables-alerter-52wx5\" (UID: \"93274098-75c7-45d2-8aa8-6329b6973bfa\") " pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:02.046145 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.046117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shpk2\" (UniqueName: \"kubernetes.io/projected/d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00-kube-api-access-shpk2\") pod \"node-resolver-pwcph\" (UID: \"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00\") " pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:02.046586 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.046555    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwsh\" (UniqueName: \"kubernetes.io/projected/ee142568-3f98-42ca-bae6-66ad70df0590-kube-api-access-lcwsh\") pod \"aws-ebs-csi-driver-node-ljv2p\" (UID: \"ee142568-3f98-42ca-bae6-66ad70df0590\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.046912 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.046869    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx94p\" (UniqueName: \"kubernetes.io/projected/2ded5c21-fead-45a6-93b9-3ff8f715f3fa-kube-api-access-xx94p\") pod \"tuned-vlhzg\" (UID: \"2ded5c21-fead-45a6-93b9-3ff8f715f3fa\") " pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.130884 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.130847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qbn6b"
Sep 10 15:16:02.141402 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.141379    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pwcph"
Sep 10 15:16:02.142051 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.142017    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ffc72e_a4eb_4f7a_be2f_e974772b84aa.slice/crio-186a478a9a608a4db49f997130d97d3dc9a2caf220737ebb1d4a42c61221e599 WatchSource:0}: Error finding container 186a478a9a608a4db49f997130d97d3dc9a2caf220737ebb1d4a42c61221e599: Status 404 returned error can't find the container with id 186a478a9a608a4db49f997130d97d3dc9a2caf220737ebb1d4a42c61221e599
Sep 10 15:16:02.152275 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.152251    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5749a66_bb3e_4dbf_bb99_fd2d9f8c9b00.slice/crio-0c287a30eeac32e9230a22d7c88fb7760a07fa09ad50bcf4a8ae5f7fc367b153 WatchSource:0}: Error finding container 0c287a30eeac32e9230a22d7c88fb7760a07fa09ad50bcf4a8ae5f7fc367b153: Status 404 returned error can't find the container with id 0c287a30eeac32e9230a22d7c88fb7760a07fa09ad50bcf4a8ae5f7fc367b153
Sep 10 15:16:02.152643 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.152624    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:02.159912 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.159700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2htvg"
Sep 10 15:16:02.162064 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.162030    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f76b831_7e7f_4a52_ba28_4336032a7d9c.slice/crio-9751c9b472c2eb72a1b6eeaa1a367db2a5c975148a49842f4d29a83650f685b7 WatchSource:0}: Error finding container 9751c9b472c2eb72a1b6eeaa1a367db2a5c975148a49842f4d29a83650f685b7: Status 404 returned error can't find the container with id 9751c9b472c2eb72a1b6eeaa1a367db2a5c975148a49842f4d29a83650f685b7
Sep 10 15:16:02.169528 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.169502    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2cfda9_d1cb_41e0_978c_8c14f669fe90.slice/crio-cb540471b1f1d391b82617ad5d6971f1a9b05f1f901dd57781b8c40c275338a9 WatchSource:0}: Error finding container cb540471b1f1d391b82617ad5d6971f1a9b05f1f901dd57781b8c40c275338a9: Status 404 returned error can't find the container with id cb540471b1f1d391b82617ad5d6971f1a9b05f1f901dd57781b8c40c275338a9
Sep 10 15:16:02.171654 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.171630    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52wx5"
Sep 10 15:16:02.180917 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.180895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fzrqw"
Sep 10 15:16:02.182305 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.182273    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93274098_75c7_45d2_8aa8_6329b6973bfa.slice/crio-262d1f9e75fe351f829fc5fda5cd0da16815159c3cc62ceae380a35927c65a9e WatchSource:0}: Error finding container 262d1f9e75fe351f829fc5fda5cd0da16815159c3cc62ceae380a35927c65a9e: Status 404 returned error can't find the container with id 262d1f9e75fe351f829fc5fda5cd0da16815159c3cc62ceae380a35927c65a9e
Sep 10 15:16:02.189599 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.189407    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p"
Sep 10 15:16:02.191438 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.191412    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e08010_50ce_42cf_96d0_a0421667d595.slice/crio-4b4de11e0816e559d09596a0d826f711c3043efc3a4467dc0ea9fe34b638a237 WatchSource:0}: Error finding container 4b4de11e0816e559d09596a0d826f711c3043efc3a4467dc0ea9fe34b638a237: Status 404 returned error can't find the container with id 4b4de11e0816e559d09596a0d826f711c3043efc3a4467dc0ea9fe34b638a237
Sep 10 15:16:02.198561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.198527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:02.200409 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.200177    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee142568_3f98_42ca_bae6_66ad70df0590.slice/crio-b0da482ff4c6b6519212971fb89f6e99f1d919de2acd9cab146b59ffe52e2737 WatchSource:0}: Error finding container b0da482ff4c6b6519212971fb89f6e99f1d919de2acd9cab146b59ffe52e2737: Status 404 returned error can't find the container with id b0da482ff4c6b6519212971fb89f6e99f1d919de2acd9cab146b59ffe52e2737
Sep 10 15:16:02.209847 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.209803    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62582b57_cacc_4c8f_8bb7_d2b2aaa19a20.slice/crio-eeef209468735dd17ea86a13e64e1082d46bca94f3bc95d6b515631bbf61e3a3 WatchSource:0}: Error finding container eeef209468735dd17ea86a13e64e1082d46bca94f3bc95d6b515631bbf61e3a3: Status 404 returned error can't find the container with id eeef209468735dd17ea86a13e64e1082d46bca94f3bc95d6b515631bbf61e3a3
Sep 10 15:16:02.219357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.219328    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:16:02.221586 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.221563    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vlhzg"
Sep 10 15:16:02.234566 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:02.234524    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ded5c21_fead_45a6_93b9_3ff8f715f3fa.slice/crio-5c528871be5e54a71f127b3d93278a7a43aeba4a50cdc492115fb5cebe88f0a5 WatchSource:0}: Error finding container 5c528871be5e54a71f127b3d93278a7a43aeba4a50cdc492115fb5cebe88f0a5: Status 404 returned error can't find the container with id 5c528871be5e54a71f127b3d93278a7a43aeba4a50cdc492115fb5cebe88f0a5
Sep 10 15:16:02.436496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.436385    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:02.436661 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.436561    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:02.436661 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.436650    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs podName:a454b8ce-8961-450d-ada9-bdb7489af486 nodeName:}" failed. No retries permitted until 2026-09-10 15:16:03.436628801 +0000 UTC m=+3.972584362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs") pod "network-metrics-daemon-t8jf6" (UID: "a454b8ce-8961-450d-ada9-bdb7489af486") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:02.537029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.536993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:02.537205 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.537066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:02.537205 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.537190    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:02.537322 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.537217    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:16:02.537322 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.537245    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:16:02.537322 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.537261    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wjggj for pod openshift-network-diagnostics/network-check-target-dhxd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:02.537322 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.537269    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret podName:75f17da5-a2cb-460c-b5ad-5804f13505dc nodeName:}" failed. No retries permitted until 2026-09-10 15:16:03.537248096 +0000 UTC m=+4.073203640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret") pod "global-pull-secret-syncer-z98zv" (UID: "75f17da5-a2cb-460c-b5ad-5804f13505dc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:02.537322 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.537315    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj podName:8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:03.537297933 +0000 UTC m=+4.073253480 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjggj" (UniqueName: "kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj") pod "network-check-target-dhxd7" (UID: "8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:02.863942 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.863825    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:11:02 +0000 UTC" deadline="2028-02-09 20:29:29.607103929 +0000 UTC"
Sep 10 15:16:02.863942 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.863891    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12413h13m26.743220052s"
Sep 10 15:16:02.948996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.948955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:02.949187 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:02.949117    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:02.961204 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.961168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vlhzg" event={"ID":"2ded5c21-fead-45a6-93b9-3ff8f715f3fa","Type":"ContainerStarted","Data":"5c528871be5e54a71f127b3d93278a7a43aeba4a50cdc492115fb5cebe88f0a5"}
Sep 10 15:16:02.962602 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.962571    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pqh9r" event={"ID":"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20","Type":"ContainerStarted","Data":"eeef209468735dd17ea86a13e64e1082d46bca94f3bc95d6b515631bbf61e3a3"}
Sep 10 15:16:02.964090 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.964061    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p" event={"ID":"ee142568-3f98-42ca-bae6-66ad70df0590","Type":"ContainerStarted","Data":"b0da482ff4c6b6519212971fb89f6e99f1d919de2acd9cab146b59ffe52e2737"}
Sep 10 15:16:02.965490 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.965442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fzrqw" event={"ID":"55e08010-50ce-42cf-96d0-a0421667d595","Type":"ContainerStarted","Data":"4b4de11e0816e559d09596a0d826f711c3043efc3a4467dc0ea9fe34b638a237"}
Sep 10 15:16:02.966751 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.966726    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2htvg" event={"ID":"da2cfda9-d1cb-41e0-978c-8c14f669fe90","Type":"ContainerStarted","Data":"cb540471b1f1d391b82617ad5d6971f1a9b05f1f901dd57781b8c40c275338a9"}
Sep 10 15:16:02.968128 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.968098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pwcph" event={"ID":"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00","Type":"ContainerStarted","Data":"0c287a30eeac32e9230a22d7c88fb7760a07fa09ad50bcf4a8ae5f7fc367b153"}
Sep 10 15:16:02.969264 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.969227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52wx5" event={"ID":"93274098-75c7-45d2-8aa8-6329b6973bfa","Type":"ContainerStarted","Data":"262d1f9e75fe351f829fc5fda5cd0da16815159c3cc62ceae380a35927c65a9e"}
Sep 10 15:16:02.970602 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.970581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"9751c9b472c2eb72a1b6eeaa1a367db2a5c975148a49842f4d29a83650f685b7"}
Sep 10 15:16:02.971752 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:02.971732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerStarted","Data":"186a478a9a608a4db49f997130d97d3dc9a2caf220737ebb1d4a42c61221e599"}
Sep 10 15:16:03.444922 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:03.444880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:03.445166 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.445108    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:03.445234 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.445175    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs podName:a454b8ce-8961-450d-ada9-bdb7489af486 nodeName:}" failed. No retries permitted until 2026-09-10 15:16:05.445156867 +0000 UTC m=+5.981112427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs") pod "network-metrics-daemon-t8jf6" (UID: "a454b8ce-8961-450d-ada9-bdb7489af486") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:03.545594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:03.545698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.545877    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.545945    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret podName:75f17da5-a2cb-460c-b5ad-5804f13505dc nodeName:}" failed. No retries permitted until 2026-09-10 15:16:05.545924641 +0000 UTC m=+6.081880187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret") pod "global-pull-secret-syncer-z98zv" (UID: "75f17da5-a2cb-460c-b5ad-5804f13505dc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.546059    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.546075    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.546088    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wjggj for pod openshift-network-diagnostics/network-check-target-dhxd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:03.546502 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.546128    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj podName:8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:05.546115736 +0000 UTC m=+6.082071282 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjggj" (UniqueName: "kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj") pod "network-check-target-dhxd7" (UID: "8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:03.864094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:03.864020    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:11:02 +0000 UTC" deadline="2028-06-01 01:19:24.589966558 +0000 UTC"
Sep 10 15:16:03.864094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:03.864065    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15106h3m20.72590654s"
Sep 10 15:16:03.954265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:03.952485    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:03.954407 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:03.952915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:03.971676 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.971635    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:03.971807 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:03.971763    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:04.948688 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:04.948603    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:04.949429 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:04.948771    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:05.000647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:04.999671    2562 generic.go:356] "Generic (PLEG): container finished" podID="330ea1231cf8707c9833237d01f0cf58" containerID="0b784d441c430159a51df66e005cbff289eb9b6ecd6263e21496b93317177959" exitCode=0
Sep 10 15:16:05.000647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:05.000323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal" event={"ID":"330ea1231cf8707c9833237d01f0cf58","Type":"ContainerDied","Data":"0b784d441c430159a51df66e005cbff289eb9b6ecd6263e21496b93317177959"}
Sep 10 15:16:05.008201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:05.008167    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal" event={"ID":"756919c48f155444a9a2e4c796c6a07b","Type":"ContainerStarted","Data":"2c79374f35928a892b008b891f5aa62971f437400068fc9f8285f8e03d5b5a0f"}
Sep 10 15:16:05.466744 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:05.466062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:05.466744 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.466260    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:05.466744 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.466328    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs podName:a454b8ce-8961-450d-ada9-bdb7489af486 nodeName:}" failed. No retries permitted until 2026-09-10 15:16:09.466309105 +0000 UTC m=+10.002264651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs") pod "network-metrics-daemon-t8jf6" (UID: "a454b8ce-8961-450d-ada9-bdb7489af486") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:05.566962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:05.567055    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.567226    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.567248    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.567262    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wjggj for pod openshift-network-diagnostics/network-check-target-dhxd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.567359    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj podName:8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:09.567339317 +0000 UTC m=+10.103294877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjggj" (UniqueName: "kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj") pod "network-check-target-dhxd7" (UID: "8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.567811    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:05.567905 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.567869    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret podName:75f17da5-a2cb-460c-b5ad-5804f13505dc nodeName:}" failed. No retries permitted until 2026-09-10 15:16:09.567850733 +0000 UTC m=+10.103806283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret") pod "global-pull-secret-syncer-z98zv" (UID: "75f17da5-a2cb-460c-b5ad-5804f13505dc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:05.952754 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:05.951944    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:05.952754 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.952086    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:05.952754 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:05.952493    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:05.952754 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:05.952604    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:06.949728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:06.949696    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:06.949910 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:06.949824    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:07.949603 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:07.949571    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:07.950040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:07.949571    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:07.950040 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:07.949723    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:07.950040 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:07.949776    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:08.948953 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:08.948846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:08.949110 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:08.949005    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:09.500502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:09.500450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:09.500964 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.500619    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:09.500964 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.500702    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs podName:a454b8ce-8961-450d-ada9-bdb7489af486 nodeName:}" failed. No retries permitted until 2026-09-10 15:16:17.500683995 +0000 UTC m=+18.036639536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs") pod "network-metrics-daemon-t8jf6" (UID: "a454b8ce-8961-450d-ada9-bdb7489af486") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:09.601410    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:09.601529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.601621    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.601651    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.601667    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wjggj for pod openshift-network-diagnostics/network-check-target-dhxd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.601667    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.601729    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj podName:8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:17.601714555 +0000 UTC m=+18.137670102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjggj" (UniqueName: "kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj") pod "network-check-target-dhxd7" (UID: "8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:09.601734 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.601749    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret podName:75f17da5-a2cb-460c-b5ad-5804f13505dc nodeName:}" failed. No retries permitted until 2026-09-10 15:16:17.601739911 +0000 UTC m=+18.137695452 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret") pod "global-pull-secret-syncer-z98zv" (UID: "75f17da5-a2cb-460c-b5ad-5804f13505dc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:09.950231 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:09.950153    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:09.950364 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.950248    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:09.950722 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:09.950703    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:09.950805 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:09.950784    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:10.949311 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:10.949269    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:10.949789 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:10.949408    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:11.953191 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:11.953158    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:11.953868 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:11.953169    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:11.953868 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:11.953363    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:11.953868 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:11.953522    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:12.948892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:12.948852    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:12.949078 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:12.948970    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:13.948810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:13.948727    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:13.949224 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:13.948729    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:13.949224 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:13.948873    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:13.949224 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:13.948915    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:14.948856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:14.948814    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:14.949236 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:14.948970    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:15.949277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:15.949244    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:15.949730 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:15.949292    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:15.949730 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:15.949373    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:15.949730 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:15.949512    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:16.949329 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:16.949293    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:16.949798 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:16.949430    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:17.566786 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:17.566746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:17.566942 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.566923    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:17.567006 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.566999    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs podName:a454b8ce-8961-450d-ada9-bdb7489af486 nodeName:}" failed. No retries permitted until 2026-09-10 15:16:33.56698234 +0000 UTC m=+34.102937880 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs") pod "network-metrics-daemon-t8jf6" (UID: "a454b8ce-8961-450d-ada9-bdb7489af486") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:17.667802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:17.667766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:17.667971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:17.667830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:17.667971 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.667941    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:17.667971 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.667961    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:16:17.668140 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.667991    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:16:17.668140 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.668003    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret podName:75f17da5-a2cb-460c-b5ad-5804f13505dc nodeName:}" failed. No retries permitted until 2026-09-10 15:16:33.6679842 +0000 UTC m=+34.203939740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret") pod "global-pull-secret-syncer-z98zv" (UID: "75f17da5-a2cb-460c-b5ad-5804f13505dc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:17.668140 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.668005    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wjggj for pod openshift-network-diagnostics/network-check-target-dhxd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:17.668140 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.668050    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj podName:8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:33.668039996 +0000 UTC m=+34.203995539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjggj" (UniqueName: "kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj") pod "network-check-target-dhxd7" (UID: "8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:17.949284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:17.949204    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:17.949467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:17.949205    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:17.949467 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.949335    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:17.949467 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:17.949389    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:18.949017 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:18.948984    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:18.949203 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:18.949136    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:19.949883 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:19.949846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:19.950270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:19.949930    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:19.950270 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:19.949955    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:19.950270 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:19.950031    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:20.949132 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:20.949113    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:20.949220 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:20.949203    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:21.949123 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:21.948907    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:21.949684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:21.948984    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:21.949684 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:21.949159    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:21.949684 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:21.949255    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:22.047746 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.047713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vlhzg" event={"ID":"2ded5c21-fead-45a6-93b9-3ff8f715f3fa","Type":"ContainerStarted","Data":"4c2b392f016c1decfc6a2ac6be65d3e458d506783e6619d44e83ff11ea70d1f5"}
Sep 10 15:16:22.049340 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.049309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pqh9r" event={"ID":"62582b57-cacc-4c8f-8bb7-d2b2aaa19a20","Type":"ContainerStarted","Data":"71c648c5abac561bf9c4cbb8ac5228fb5f727fcb39a18b84dcb291d6e4b6ef57"}
Sep 10 15:16:22.050978 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.050955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p" event={"ID":"ee142568-3f98-42ca-bae6-66ad70df0590","Type":"ContainerStarted","Data":"2b126356a7fcee1c9b70310f30663afed0359ec0d8b1e7251e7f0c81d0c1b352"}
Sep 10 15:16:22.052629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.052604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fzrqw" event={"ID":"55e08010-50ce-42cf-96d0-a0421667d595","Type":"ContainerStarted","Data":"fe59ef557420dd7c5edb8782ff5a0b3e339a78efeb0bc7ab390da8490a6096f3"}
Sep 10 15:16:22.054288 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.054263    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2htvg" event={"ID":"da2cfda9-d1cb-41e0-978c-8c14f669fe90","Type":"ContainerStarted","Data":"56f0b44918df03bd9f853ade2f1a16564383a1013755aa82c739e34b61333060"}
Sep 10 15:16:22.055801 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.055772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pwcph" event={"ID":"d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00","Type":"ContainerStarted","Data":"a1363f28921a8aadffab0d6cf4dcdfa1d13c7f7ada59e0344fcb92da0b7c02f6"}
Sep 10 15:16:22.057859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.057834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal" event={"ID":"330ea1231cf8707c9833237d01f0cf58","Type":"ContainerStarted","Data":"230e7091b691a59cd6ef1fbb069d0afeec7f23087cfca09e3594f067ef9b0bf4"}
Sep 10 15:16:22.060739 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.060722    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:16:22.061100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.061080    2562 generic.go:356] "Generic (PLEG): container finished" podID="7f76b831-7e7f-4a52-ba28-4336032a7d9c" containerID="f75b9e573b22aad18cccf8d44ae7c0ba6375d96fc5932f4a05e90711f5cc0daa" exitCode=1
Sep 10 15:16:22.061153 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.061140    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"add0aaa5814bc4d2838b20acd18805ccbf2a5e73fc95d613410fcc8c65727caa"}
Sep 10 15:16:22.061207 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.061194    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"21411467a635942ecc0777dfd7860bc917ee221a7123de20850105e475cc99f4"}
Sep 10 15:16:22.061258 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.061231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"c9490ee588ec4d5cdcdec5b9419d834e316a83c3192232111ff040f8e8f02d33"}
Sep 10 15:16:22.061308 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.061280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"c8f6deeae15f2f906caead7533874e10b4871c061d6bdb75a0e64367dce2f82f"}
Sep 10 15:16:22.061354 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.061327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerDied","Data":"f75b9e573b22aad18cccf8d44ae7c0ba6375d96fc5932f4a05e90711f5cc0daa"}
Sep 10 15:16:22.061415 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.061376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"17d2631ea1036b2a5f2c92802d0241cc7ab2759bdbff965aaa684a7eb39d665f"}
Sep 10 15:16:22.062799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.062772    2562 generic.go:356] "Generic (PLEG): container finished" podID="51ffc72e-a4eb-4f7a-be2f-e974772b84aa" containerID="82a30f73756f6ebdd895709259e2c37c023a641974447bd0dcf2ad000ef0460e" exitCode=0
Sep 10 15:16:22.062895 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.062799    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerDied","Data":"82a30f73756f6ebdd895709259e2c37c023a641974447bd0dcf2ad000ef0460e"}
Sep 10 15:16:22.070740 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.070702    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-193.ec2.internal" podStartSLOduration=20.070689529 podStartE2EDuration="20.070689529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:05.048932808 +0000 UTC m=+5.584888383" watchObservedRunningTime="2026-09-10 15:16:22.070689529 +0000 UTC m=+22.606645094"
Sep 10 15:16:22.070843 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.070776    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vlhzg" podStartSLOduration=2.345066999 podStartE2EDuration="21.070768418s" totalImagesPullingTime="18.725701419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:22.070030484 +0000 UTC m=+22.605986047" watchObservedRunningTime="2026-09-10 15:16:22.070768418 +0000 UTC m=+22.606723981"
Sep 10 15:16:22.086871 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.086826    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2htvg" podStartSLOduration=3.295438969 podStartE2EDuration="22.086815111s" totalImagesPullingTime="18.791376142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:22.086191787 +0000 UTC m=+22.622147360" watchObservedRunningTime="2026-09-10 15:16:22.086815111 +0000 UTC m=+22.622770672"
Sep 10 15:16:22.132225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.132168    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fzrqw" podStartSLOduration=2.03925662 podStartE2EDuration="21.132153798s" totalImagesPullingTime="19.092897178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:22.108365416 +0000 UTC m=+22.644320997" watchObservedRunningTime="2026-09-10 15:16:22.132153798 +0000 UTC m=+22.668109360"
Sep 10 15:16:22.152175 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.152130    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pqh9r" podStartSLOduration=6.286834774 podStartE2EDuration="21.152114172s" totalImagesPullingTime="14.865279398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:22.132720204 +0000 UTC m=+22.668675787" watchObservedRunningTime="2026-09-10 15:16:22.152114172 +0000 UTC m=+22.688069735"
Sep 10 15:16:22.152362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.152330    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-193.ec2.internal" podStartSLOduration=20.15232304 podStartE2EDuration="20.15232304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:22.152259512 +0000 UTC m=+22.688215056" watchObservedRunningTime="2026-09-10 15:16:22.15232304 +0000 UTC m=+22.688278616"
Sep 10 15:16:22.195112 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.195072    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pwcph" podStartSLOduration=3.401972703 podStartE2EDuration="22.19505904s" totalImagesPullingTime="18.793086337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:22.194155992 +0000 UTC m=+22.730111554" watchObservedRunningTime="2026-09-10 15:16:22.19505904 +0000 UTC m=+22.731014602"
Sep 10 15:16:22.198068 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.198044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:22.258174 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.258146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:22.357915 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.357876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:22.541882 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.541858    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 15:16:22.894620 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.894471    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T15:16:22.541877803Z","UUID":"23cb646a-47de-4dd5-bb69-034ed8a0d37e","Handler":null,"Name":"","Endpoint":""}
Sep 10 15:16:22.896218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.896190    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 15:16:22.896218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.896224    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 15:16:22.949286 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:22.949259    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:22.950032 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:22.949390    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:23.070493 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:23.070443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p" event={"ID":"ee142568-3f98-42ca-bae6-66ad70df0590","Type":"ContainerStarted","Data":"1244015b50d59a247b838a9a34cf4e39f69c1db2ace0873c82d6478c298c2cdc"}
Sep 10 15:16:23.072315 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:23.072284    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52wx5" event={"ID":"93274098-75c7-45d2-8aa8-6329b6973bfa","Type":"ContainerStarted","Data":"818c4e26004d7661fcb45784b17dbfe56a12f826daf89aa450398ff8b1c08929"}
Sep 10 15:16:23.119275 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:23.119223    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-52wx5" podStartSLOduration=3.339234063 podStartE2EDuration="22.119207729s" totalImagesPullingTime="18.779973666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:23.110823175 +0000 UTC m=+23.646778763" watchObservedRunningTime="2026-09-10 15:16:23.119207729 +0000 UTC m=+23.655163291"
Sep 10 15:16:23.258269 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:23.258193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:23.365142 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:23.365101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:23.949018 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:23.948931    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:23.949168 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:23.948934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:23.949168 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:23.949052    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:23.949168 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:23.949144    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:24.077930 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:24.077904    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:16:24.078376 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:24.078346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"683a5554b026859422632267a1355442839b2e10b36eb876423b25dd9b263d6a"}
Sep 10 15:16:24.080668 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:24.080640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p" event={"ID":"ee142568-3f98-42ca-bae6-66ad70df0590","Type":"ContainerStarted","Data":"6a400de9ec987eb0161ce5ac17a64f3ba1c090945902cde07ed7b340bdc914fb"}
Sep 10 15:16:24.100067 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:24.100030    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ljv2p" podStartSLOduration=2.038705822 podStartE2EDuration="23.100014262s" totalImagesPullingTime="21.06130844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:24.099898319 +0000 UTC m=+24.635853882" watchObservedRunningTime="2026-09-10 15:16:24.100014262 +0000 UTC m=+24.635969826"
Sep 10 15:16:24.949033 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:24.948863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:24.949208 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:24.949135    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:25.754188 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:25.754156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:25.855447 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:25.855413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:25.922224 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:25.922189    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:25.922841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:25.922819    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:25.944046 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:25.944017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:25.948749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:25.948727    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:25.948850 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:25.948734    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:25.948850 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:25.948823    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:25.948950 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:25.948927    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:26.087388 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:26.087311    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:26.087892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:26.087871    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pqh9r"
Sep 10 15:16:26.156940 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:26.156905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:26.949186 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:26.949154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:26.949561 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:26.949264    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:27.094259 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:27.094187    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:16:27.094755 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:27.094648    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"46a6611f8b799a3b3a5827869590f35a0e0e12b886c51245e5ef3d51cfcba951"}
Sep 10 15:16:27.095267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:27.095149    2562 scope.go:117] "RemoveContainer" containerID="f75b9e573b22aad18cccf8d44ae7c0ba6375d96fc5932f4a05e90711f5cc0daa"
Sep 10 15:16:27.949561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:27.949531    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:27.950057 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:27.949629    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:27.950057 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:27.949734    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:27.950057 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:27.949852    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:28.100429 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.100403    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:16:28.100795 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.100767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" event={"ID":"7f76b831-7e7f-4a52-ba28-4336032a7d9c","Type":"ContainerStarted","Data":"ce44f3f7574f65457afa61f155704b0ce84d8e98e38e4e999e46d83b38bde4ff"}
Sep 10 15:16:28.100893 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.100833    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 15:16:28.101239 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.101208    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:28.101370 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.101283    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:28.102810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.102786    2562 generic.go:356] "Generic (PLEG): container finished" podID="51ffc72e-a4eb-4f7a-be2f-e974772b84aa" containerID="e365116dbef56a0b73aff6b8169187f15fbcc6ab3a30b66cdb0c5379abf9c7f2" exitCode=0
Sep 10 15:16:28.102931 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.102873    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerDied","Data":"e365116dbef56a0b73aff6b8169187f15fbcc6ab3a30b66cdb0c5379abf9c7f2"}
Sep 10 15:16:28.116027 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.115998    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:28.116155 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.116143    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:28.132602 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.132563    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx" podStartSLOduration=9.312672129 podStartE2EDuration="28.132551523s" totalImagesPullingTime="18.819879394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:28.13131606 +0000 UTC m=+28.667271634" watchObservedRunningTime="2026-09-10 15:16:28.132551523 +0000 UTC m=+28.668507084"
Sep 10 15:16:28.947469 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.947374    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dhxd7"]
Sep 10 15:16:28.947666 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.947513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:28.947666 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:28.947628    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:28.948677 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.948657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:28.948787 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:28.948762    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:28.950467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.950434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z98zv"]
Sep 10 15:16:28.951067 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.950524    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:28.951067 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:28.950603    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:28.951148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:28.951066    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t8jf6"]
Sep 10 15:16:29.105420 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:29.105384    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:29.105606 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:29.105445    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 15:16:29.105606 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:29.105548    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:29.673837 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:29.673746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:29.761682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:29.761653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:30.108979 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:30.108899    2562 generic.go:356] "Generic (PLEG): container finished" podID="51ffc72e-a4eb-4f7a-be2f-e974772b84aa" containerID="981909658f17190778ea1e6ec1d437875d2474e26e66f888602eacd426e88b1f" exitCode=0
Sep 10 15:16:30.109410 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:30.108979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerDied","Data":"981909658f17190778ea1e6ec1d437875d2474e26e66f888602eacd426e88b1f"}
Sep 10 15:16:30.109410 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:30.109060    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 15:16:30.908219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:30.908179    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:16:30.949026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:30.948917    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:30.949026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:30.948949    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:30.949026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:30.948949    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:30.949324 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:30.949048    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:30.949324 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:30.949099    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:30.949324 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:30.949153    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:32.115423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:32.115391    2562 generic.go:356] "Generic (PLEG): container finished" podID="51ffc72e-a4eb-4f7a-be2f-e974772b84aa" containerID="e88ea95ac5de64521e9434140570478b7ef748182729045146c0fd10089cd689" exitCode=0
Sep 10 15:16:32.115848 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:32.115467    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerDied","Data":"e88ea95ac5de64521e9434140570478b7ef748182729045146c0fd10089cd689"}
Sep 10 15:16:32.948661 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:32.948582    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:32.948661 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:32.948584    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:32.948867 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:32.948582    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:32.948867 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:32.948700    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:32.948867 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:32.948806    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:32.949000 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:32.948864    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:33.585184 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:33.585144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:33.585658 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.585299    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:33.585658 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.585373    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs podName:a454b8ce-8961-450d-ada9-bdb7489af486 nodeName:}" failed. No retries permitted until 2026-09-10 15:17:05.585357197 +0000 UTC m=+66.121312751 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs") pod "network-metrics-daemon-t8jf6" (UID: "a454b8ce-8961-450d-ada9-bdb7489af486") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:16:33.686250 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:33.686210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:33.686474 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:33.686282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:33.686474 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.686388    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:33.686474 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.686438    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret podName:75f17da5-a2cb-460c-b5ad-5804f13505dc nodeName:}" failed. No retries permitted until 2026-09-10 15:17:05.686424863 +0000 UTC m=+66.222380417 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret") pod "global-pull-secret-syncer-z98zv" (UID: "75f17da5-a2cb-460c-b5ad-5804f13505dc") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:16:33.686474 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.686389    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:16:33.686710 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.686492    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:16:33.686710 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.686506    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wjggj for pod openshift-network-diagnostics/network-check-target-dhxd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:33.686710 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:33.686562    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj podName:8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d nodeName:}" failed. No retries permitted until 2026-09-10 15:17:05.686544951 +0000 UTC m=+66.222500525 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjggj" (UniqueName: "kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj") pod "network-check-target-dhxd7" (UID: "8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:16:34.949678 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:34.949614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:34.950057 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:34.949615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:34.950057 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:34.949728    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhxd7" podUID="8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d"
Sep 10 15:16:34.950057 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:34.949816    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t8jf6" podUID="a454b8ce-8961-450d-ada9-bdb7489af486"
Sep 10 15:16:34.950057 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:34.949615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:34.950057 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:34.949901    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z98zv" podUID="75f17da5-a2cb-460c-b5ad-5804f13505dc"
Sep 10 15:16:35.184624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.184592    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-193.ec2.internal" event="NodeReady"
Sep 10 15:16:35.184776 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.184741    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 15:16:35.238821 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.238714    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x5bkx"]
Sep 10 15:16:35.242603 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.242580    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hc4nl"]
Sep 10 15:16:35.242736 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.242714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.245532 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.245512    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:35.245743 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.245721    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 15:16:35.245892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.245862    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 15:16:35.246005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.245926    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4q2px\""
Sep 10 15:16:35.247997 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.247978    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 15:16:35.248279 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.248259    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 15:16:35.248709 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.248602    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-46mx6\""
Sep 10 15:16:35.248878 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.248858    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 15:16:35.251957 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.251859    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hc4nl"]
Sep 10 15:16:35.258889 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.258860    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x5bkx"]
Sep 10 15:16:35.365373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.365335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:35.398190 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.398155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8067dd20-4892-4078-aef9-b405fd646a7b-cert\") pod \"ingress-canary-hc4nl\" (UID: \"8067dd20-4892-4078-aef9-b405fd646a7b\") " pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:35.398190 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.398191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px764\" (UniqueName: \"kubernetes.io/projected/8067dd20-4892-4078-aef9-b405fd646a7b-kube-api-access-px764\") pod \"ingress-canary-hc4nl\" (UID: \"8067dd20-4892-4078-aef9-b405fd646a7b\") " pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:35.398419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.398210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcf4w\" (UniqueName: \"kubernetes.io/projected/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-kube-api-access-qcf4w\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.398419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.398272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-tmp-dir\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.398419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.398306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-config-volume\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.398419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.398334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.430807 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.430777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:35.498911 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.498828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8067dd20-4892-4078-aef9-b405fd646a7b-cert\") pod \"ingress-canary-hc4nl\" (UID: \"8067dd20-4892-4078-aef9-b405fd646a7b\") " pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:35.498911 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.498868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px764\" (UniqueName: \"kubernetes.io/projected/8067dd20-4892-4078-aef9-b405fd646a7b-kube-api-access-px764\") pod \"ingress-canary-hc4nl\" (UID: \"8067dd20-4892-4078-aef9-b405fd646a7b\") " pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:35.498911 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.498887    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcf4w\" (UniqueName: \"kubernetes.io/projected/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-kube-api-access-qcf4w\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.499180 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.498938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-tmp-dir\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.499180 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.498968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-config-volume\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.499180 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.498989    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.499180 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:35.499073    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:16:35.499180 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:35.499127    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls podName:4983b7b7-1ffe-45a5-8cdf-117d59ec203a nodeName:}" failed. No retries permitted until 2026-09-10 15:16:35.999110368 +0000 UTC m=+36.535065924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls") pod "dns-default-x5bkx" (UID: "4983b7b7-1ffe-45a5-8cdf-117d59ec203a") : secret "dns-default-metrics-tls" not found
Sep 10 15:16:35.499428 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.499277    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-tmp-dir\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.500523 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.500498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-config-volume\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.503565 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.503543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8067dd20-4892-4078-aef9-b405fd646a7b-cert\") pod \"ingress-canary-hc4nl\" (UID: \"8067dd20-4892-4078-aef9-b405fd646a7b\") " pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:35.507988 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.507967    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px764\" (UniqueName: \"kubernetes.io/projected/8067dd20-4892-4078-aef9-b405fd646a7b-kube-api-access-px764\") pod \"ingress-canary-hc4nl\" (UID: \"8067dd20-4892-4078-aef9-b405fd646a7b\") " pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:35.508085 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.508066    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcf4w\" (UniqueName: \"kubernetes.io/projected/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-kube-api-access-qcf4w\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:35.565218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:35.565185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hc4nl"
Sep 10 15:16:36.003401 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.003362    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:36.003803 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:36.003521    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:16:36.003803 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:36.003582    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls podName:4983b7b7-1ffe-45a5-8cdf-117d59ec203a nodeName:}" failed. No retries permitted until 2026-09-10 15:16:37.003563314 +0000 UTC m=+37.539518853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls") pod "dns-default-x5bkx" (UID: "4983b7b7-1ffe-45a5-8cdf-117d59ec203a") : secret "dns-default-metrics-tls" not found
Sep 10 15:16:36.671891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.671671    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hc4nl"]
Sep 10 15:16:36.679417 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:36.679388    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8067dd20_4892_4078_aef9_b405fd646a7b.slice/crio-7c6ecc4421533526406e1f032a3e852a3bdf62b99d9932998bec4f6b8ef6da49 WatchSource:0}: Error finding container 7c6ecc4421533526406e1f032a3e852a3bdf62b99d9932998bec4f6b8ef6da49: Status 404 returned error can't find the container with id 7c6ecc4421533526406e1f032a3e852a3bdf62b99d9932998bec4f6b8ef6da49
Sep 10 15:16:36.948676 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.948591    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:16:36.948811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.948684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:16:36.948811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.948765    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:16:36.952480 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.952442    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-96bxt\""
Sep 10 15:16:36.952480 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.952473    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 15:16:36.952643 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.952481    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 15:16:36.952643 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.952519    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 15:16:36.952643 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.952533    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5wmh\""
Sep 10 15:16:36.952818 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:36.952804    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 15:16:37.012132 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:37.012106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:37.012688 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:37.012223    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:16:37.012688 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:37.012277    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls podName:4983b7b7-1ffe-45a5-8cdf-117d59ec203a nodeName:}" failed. No retries permitted until 2026-09-10 15:16:39.01226479 +0000 UTC m=+39.548220343 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls") pod "dns-default-x5bkx" (UID: "4983b7b7-1ffe-45a5-8cdf-117d59ec203a") : secret "dns-default-metrics-tls" not found
Sep 10 15:16:37.131096 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:37.131061    2562 generic.go:356] "Generic (PLEG): container finished" podID="51ffc72e-a4eb-4f7a-be2f-e974772b84aa" containerID="691611fd5dd7e5cc111f836fa4e362d246d82af68630e281367ac7098e27378b" exitCode=0
Sep 10 15:16:37.131263 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:37.131148    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerDied","Data":"691611fd5dd7e5cc111f836fa4e362d246d82af68630e281367ac7098e27378b"}
Sep 10 15:16:37.132337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:37.132313    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hc4nl" event={"ID":"8067dd20-4892-4078-aef9-b405fd646a7b","Type":"ContainerStarted","Data":"7c6ecc4421533526406e1f032a3e852a3bdf62b99d9932998bec4f6b8ef6da49"}
Sep 10 15:16:37.861305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:37.861273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.139001 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.138924    2562 generic.go:356] "Generic (PLEG): container finished" podID="51ffc72e-a4eb-4f7a-be2f-e974772b84aa" containerID="6e3fc2cd217bd5c2348656276cfb5b0ab386686c1e6286dd84e0630ed8c3a0be" exitCode=0
Sep 10 15:16:38.139001 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.138989    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerDied","Data":"6e3fc2cd217bd5c2348656276cfb5b0ab386686c1e6286dd84e0630ed8c3a0be"}
Sep 10 15:16:38.322749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.322718    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p8sqc"]
Sep 10 15:16:38.322913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.322877    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.330048 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.330023    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.333058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.333032    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 15:16:38.333219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.333202    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 15:16:38.333321 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.333302    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 15:16:38.333380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.333309    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 15:16:38.333483 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.333445    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-696sd\""
Sep 10 15:16:38.337123 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.337102    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8sqc"]
Sep 10 15:16:38.389867 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.389811    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.422955 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.422875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.423120 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.422966    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9kn7\" (UniqueName: \"kubernetes.io/projected/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-api-access-r9kn7\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.423120 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.422996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-data-volume\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.423120 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.423013    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.423120 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.423036    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-crio-socket\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.445360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.445331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.523546 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.523505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.523734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.523564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9kn7\" (UniqueName: \"kubernetes.io/projected/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-api-access-r9kn7\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.523734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.523601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-data-volume\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.523734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.523627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.523734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.523653    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-crio-socket\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.523734 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:38.523658    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:16:38.523979 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:38.523741    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:39.023719572 +0000 UTC m=+39.559675139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : configmap "kube-rbac-proxy" not found
Sep 10 15:16:38.523979 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:38.523835    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:38.523979 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:38.523878    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:39.023864862 +0000 UTC m=+39.559820412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:38.523979 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.523941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-crio-socket\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.523979 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.523971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-data-volume\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.533947 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.533925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9kn7\" (UniqueName: \"kubernetes.io/projected/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-api-access-r9kn7\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:38.546794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.546769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.646625 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.646540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.749903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.749874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.837181 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.837145    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"]
Sep 10 15:16:38.837962 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.837932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.858782 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.858756    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"]
Sep 10 15:16:38.858920 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.858869    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"
Sep 10 15:16:38.861548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.861517    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 15:16:38.861668 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.861586    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vt477\""
Sep 10 15:16:38.861805 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.861789    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 15:16:38.906151 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.906079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:38.964841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:38.964807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:39.028403 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.028358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:39.028568 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.028416    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d22gz\" (UniqueName: \"kubernetes.io/projected/629b4e6c-3fb5-4b92-9b56-de129175adfe-kube-api-access-d22gz\") pod \"migrator-fdbb8fdff-4d89n\" (UID: \"629b4e6c-3fb5-4b92-9b56-de129175adfe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"
Sep 10 15:16:39.028568 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.028516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:39.028679 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.028589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:39.028679 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:39.028641    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:16:39.028754 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:39.028696    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:16:39.028754 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:39.028725    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:39.028754 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:39.028703    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls podName:4983b7b7-1ffe-45a5-8cdf-117d59ec203a nodeName:}" failed. No retries permitted until 2026-09-10 15:16:43.028682519 +0000 UTC m=+43.564638073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls") pod "dns-default-x5bkx" (UID: "4983b7b7-1ffe-45a5-8cdf-117d59ec203a") : secret "dns-default-metrics-tls" not found
Sep 10 15:16:39.028876 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:39.028771    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:40.028758278 +0000 UTC m=+40.564713817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : configmap "kube-rbac-proxy" not found
Sep 10 15:16:39.028876 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:39.028789    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:40.028779922 +0000 UTC m=+40.564735461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:39.066368 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.066331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:39.129484 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.129438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d22gz\" (UniqueName: \"kubernetes.io/projected/629b4e6c-3fb5-4b92-9b56-de129175adfe-kube-api-access-d22gz\") pod \"migrator-fdbb8fdff-4d89n\" (UID: \"629b4e6c-3fb5-4b92-9b56-de129175adfe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"
Sep 10 15:16:39.137986 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.137955    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d22gz\" (UniqueName: \"kubernetes.io/projected/629b4e6c-3fb5-4b92-9b56-de129175adfe-kube-api-access-d22gz\") pod \"migrator-fdbb8fdff-4d89n\" (UID: \"629b4e6c-3fb5-4b92-9b56-de129175adfe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"
Sep 10 15:16:39.145139 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.145117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" event={"ID":"51ffc72e-a4eb-4f7a-be2f-e974772b84aa","Type":"ContainerStarted","Data":"df365622d68c47f1b7b54887120755f5f5c059bbf4738c59511ec431af118080"}
Sep 10 15:16:39.169955 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.169908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"
Sep 10 15:16:39.179468 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.179429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:39.186712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.186672    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qbn6b" podStartSLOduration=8.557024499 podStartE2EDuration="39.186657702s" totalImagesPullingTime="30.629633203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:39.182249251 +0000 UTC m=+39.718204834" watchObservedRunningTime="2026-09-10 15:16:39.186657702 +0000 UTC m=+39.722613263"
Sep 10 15:16:39.365895 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.365854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:39.447998 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.447928    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n"]
Sep 10 15:16:39.451161 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:39.451135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:39.503924 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:39.503895    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629b4e6c_3fb5_4b92_9b56_de129175adfe.slice/crio-68ef47c0cd6224f6a1363133e4e7c8da2b58b3946588f9bd0d60ef0e28dcd24f WatchSource:0}: Error finding container 68ef47c0cd6224f6a1363133e4e7c8da2b58b3946588f9bd0d60ef0e28dcd24f: Status 404 returned error can't find the container with id 68ef47c0cd6224f6a1363133e4e7c8da2b58b3946588f9bd0d60ef0e28dcd24f
Sep 10 15:16:40.036698 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.036658    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:40.036886 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.036721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:40.036886 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:40.036785    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:16:40.036886 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:40.036827    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:40.036886 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:40.036849    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:42.036832776 +0000 UTC m=+42.572788333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : configmap "kube-rbac-proxy" not found
Sep 10 15:16:40.036886 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:40.036880    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:42.036867066 +0000 UTC m=+42.572822619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:40.150149 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.150089    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hc4nl" event={"ID":"8067dd20-4892-4078-aef9-b405fd646a7b","Type":"ContainerStarted","Data":"238d7ef49efef1acc900e2b81273be09795503bf9d13c04dfac175d97c745a2f"}
Sep 10 15:16:40.151466 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.151410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n" event={"ID":"629b4e6c-3fb5-4b92-9b56-de129175adfe","Type":"ContainerStarted","Data":"68ef47c0cd6224f6a1363133e4e7c8da2b58b3946588f9bd0d60ef0e28dcd24f"}
Sep 10 15:16:40.167779 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.167730    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hc4nl" podStartSLOduration=2.317142265 podStartE2EDuration="5.167712782s" totalImagesPullingTime="2.850570517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:40.166850553 +0000 UTC m=+40.702806126" watchObservedRunningTime="2026-09-10 15:16:40.167712782 +0000 UTC m=+40.703668343"
Sep 10 15:16:40.446159 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.446130    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pwcph_d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00/dns-node-resolver/0.log"
Sep 10 15:16:40.912075 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.912041    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-lgz7t"]
Sep 10 15:16:40.924937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.924905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-lgz7t"]
Sep 10 15:16:40.925079 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.925012    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:40.927764 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.927740    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fzbxg\""
Sep 10 15:16:40.927764 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.927751    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 15:16:40.928722 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.928689    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 15:16:40.928722 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.928701    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 15:16:40.928902 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.928740    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 15:16:40.951632 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:40.951601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:41.005542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.005517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:41.046917 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.046889    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/af6ebbb2-ae51-4796-aa15-78671e294ff0-signing-key\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.046917 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.046919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpwk9\" (UniqueName: \"kubernetes.io/projected/af6ebbb2-ae51-4796-aa15-78671e294ff0-kube-api-access-fpwk9\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.047111 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.046982    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2htvg_da2cfda9-d1cb-41e0-978c-8c14f669fe90/node-ca/0.log"
Sep 10 15:16:41.047111 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.047053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/af6ebbb2-ae51-4796-aa15-78671e294ff0-signing-cabundle\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.147577 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.147540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/af6ebbb2-ae51-4796-aa15-78671e294ff0-signing-cabundle\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.147735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.147595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/af6ebbb2-ae51-4796-aa15-78671e294ff0-signing-key\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.147735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.147692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpwk9\" (UniqueName: \"kubernetes.io/projected/af6ebbb2-ae51-4796-aa15-78671e294ff0-kube-api-access-fpwk9\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.148342 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.148316    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/af6ebbb2-ae51-4796-aa15-78671e294ff0-signing-cabundle\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.149943 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.149915    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/af6ebbb2-ae51-4796-aa15-78671e294ff0-signing-key\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.156173 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.156153    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpwk9\" (UniqueName: \"kubernetes.io/projected/af6ebbb2-ae51-4796-aa15-78671e294ff0-kube-api-access-fpwk9\") pod \"service-ca-75d99c484-lgz7t\" (UID: \"af6ebbb2-ae51-4796-aa15-78671e294ff0\") " pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.234419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.234348    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-lgz7t"
Sep 10 15:16:41.531533 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.529639    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-lgz7t"]
Sep 10 15:16:41.532755 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:16:41.532729    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6ebbb2_ae51_4796_aa15_78671e294ff0.slice/crio-5aa5e3bfdfe6d8f4273370ad79d4d86292b52af088af3f141f0867723512d8b9 WatchSource:0}: Error finding container 5aa5e3bfdfe6d8f4273370ad79d4d86292b52af088af3f141f0867723512d8b9: Status 404 returned error can't find the container with id 5aa5e3bfdfe6d8f4273370ad79d4d86292b52af088af3f141f0867723512d8b9
Sep 10 15:16:41.848072 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:41.848046    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hc4nl_8067dd20-4892-4078-aef9-b405fd646a7b/serve-healthcheck-canary/0.log"
Sep 10 15:16:42.006873 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:42.006841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:42.053970 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:42.053929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:42.054141 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:42.054013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:42.054141 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:42.054053    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:16:42.054141 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:42.054133    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:46.05411645 +0000 UTC m=+46.590072005 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : configmap "kube-rbac-proxy" not found
Sep 10 15:16:42.054269 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:42.054151    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:42.054269 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:42.054200    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:46.054187246 +0000 UTC m=+46.590142821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:42.158982 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:42.158905    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n" event={"ID":"629b4e6c-3fb5-4b92-9b56-de129175adfe","Type":"ContainerStarted","Data":"693f208c3aa83693208d173e766c0ba5323d344e7f076b30c1de0a16bead1a64"}
Sep 10 15:16:42.158982 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:42.158968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n" event={"ID":"629b4e6c-3fb5-4b92-9b56-de129175adfe","Type":"ContainerStarted","Data":"3b4b887b0d949186b365deb40e2c968813559ad239a796e76e6a9f6013bd0f89"}
Sep 10 15:16:42.160465 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:42.160422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-lgz7t" event={"ID":"af6ebbb2-ae51-4796-aa15-78671e294ff0","Type":"ContainerStarted","Data":"5aa5e3bfdfe6d8f4273370ad79d4d86292b52af088af3f141f0867723512d8b9"}
Sep 10 15:16:42.180287 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:42.180238    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4d89n" podStartSLOduration=2.446984215 podStartE2EDuration="4.180223016s" totalImagesPullingTime="1.733238801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:42.179998609 +0000 UTC m=+42.715954173" watchObservedRunningTime="2026-09-10 15:16:42.180223016 +0000 UTC m=+42.716178577"
Sep 10 15:16:43.062829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:43.062791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:43.063012 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:43.062941    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:16:43.063012 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:43.063005    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls podName:4983b7b7-1ffe-45a5-8cdf-117d59ec203a nodeName:}" failed. No retries permitted until 2026-09-10 15:16:51.062987566 +0000 UTC m=+51.598943119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls") pod "dns-default-x5bkx" (UID: "4983b7b7-1ffe-45a5-8cdf-117d59ec203a") : secret "dns-default-metrics-tls" not found
Sep 10 15:16:44.167252 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:44.167166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-lgz7t" event={"ID":"af6ebbb2-ae51-4796-aa15-78671e294ff0","Type":"ContainerStarted","Data":"47bf2439a98dc91cd056dfae81c17d31e7d2bb2ac779a52d4d60eaec5ae7b03b"}
Sep 10 15:16:44.189433 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:44.189280    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-lgz7t" podStartSLOduration=1.874654775 podStartE2EDuration="4.189261655s" totalImagesPullingTime="2.31460688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:44.189070314 +0000 UTC m=+44.725025877" watchObservedRunningTime="2026-09-10 15:16:44.189261655 +0000 UTC m=+44.725217219"
Sep 10 15:16:44.719798 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:44.719762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:46.084094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:46.084004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:46.084094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:46.084068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:46.084553 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:46.084127    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:16:46.084553 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:46.084190    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:54.084175928 +0000 UTC m=+54.620131471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : configmap "kube-rbac-proxy" not found
Sep 10 15:16:46.084553 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:46.084256    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:46.084553 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:46.084313    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:16:54.084299647 +0000 UTC m=+54.620255186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:46.585656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:46.585625    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:46.705587 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:46.705548    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:46.888943 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:46.888856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:46.988711 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:46.988661    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:47.129061 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:47.129025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:47.339745 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:47.339712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:48.133861 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:48.133835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:48.328066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:48.328033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:48.484354 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:48.484275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:48.901356 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:48.901321    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:49.287358 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:49.287271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:49.485482 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:49.485410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:50.127385 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:50.127353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:50.479843 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:50.479753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:50.533821 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:50.533778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:50.888165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:50.888126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:50.990579 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:50.990541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:51.089667 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:51.089635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:51.123397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:51.123364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:51.125597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:51.125569    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4983b7b7-1ffe-45a5-8cdf-117d59ec203a-metrics-tls\") pod \"dns-default-x5bkx\" (UID: \"4983b7b7-1ffe-45a5-8cdf-117d59ec203a\") " pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:51.156605 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:51.156539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:51.453221 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:51.449415    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x5bkx"]
Sep 10 15:16:51.466636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:51.466607    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:51.567738 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:51.567702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:52.086004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:52.085969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:52.188393 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:52.188358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x5bkx" event={"ID":"4983b7b7-1ffe-45a5-8cdf-117d59ec203a","Type":"ContainerStarted","Data":"db5053078b42e63a759cbefed502fa5d012795d6a551f9a1d9156c75c784a30f"}
Sep 10 15:16:52.487523 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:52.487415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:52.699752 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:52.699711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:52.883049 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:52.882995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:53.284809 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:53.284725    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:53.483282 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:53.483256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:53.890388 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:53.890340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:54.147717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.147626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:54.147717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.147675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:54.147904 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:54.147774    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:54.147904 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:16:54.147848    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls podName:7182f42f-a1d6-405e-bf4d-f6ffee314b6d nodeName:}" failed. No retries permitted until 2026-09-10 15:17:10.147832463 +0000 UTC m=+70.683788003 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8sqc" (UID: "7182f42f-a1d6-405e-bf4d-f6ffee314b6d") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:16:54.148177 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.148158    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:16:54.196202 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.196171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x5bkx" event={"ID":"4983b7b7-1ffe-45a5-8cdf-117d59ec203a","Type":"ContainerStarted","Data":"31f49032d4f300d94f2e1885faf324e824459b81023b87c38f1a04f2328673d5"}
Sep 10 15:16:54.196319 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.196218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x5bkx" event={"ID":"4983b7b7-1ffe-45a5-8cdf-117d59ec203a","Type":"ContainerStarted","Data":"f05a22326b7bef5af54e9d9905c4e63aefb2faf74a35eca8946da99ea48aec0a"}
Sep 10 15:16:54.196319 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.196248    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:16:54.215080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.215034    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x5bkx" podStartSLOduration=17.293017239 podStartE2EDuration="19.215018878s" totalImagesPullingTime="1.922001639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:54.214411189 +0000 UTC m=+54.750366771" watchObservedRunningTime="2026-09-10 15:16:54.215018878 +0000 UTC m=+54.750974439"
Sep 10 15:16:54.484239 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:54.484154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:55.084339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:55.084302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:55.292216 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:55.292176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:55.483667 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:55.483590    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:55.889774 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:55.889740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:56.484679 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:56.484647    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:56.884599 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:56.884562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:57.082601 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:57.082568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:16:57.288151 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:16:57.288070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:01.122175 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:01.122147    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dn2vx"
Sep 10 15:17:03.756080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.756043    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-b2znc"]
Sep 10 15:17:03.758521 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.758427    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:03.762049 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.762032    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 15:17:03.762049 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.762034    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v77tp\""
Sep 10 15:17:03.762191 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.762109    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 15:17:03.773648 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.773620    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-b2znc"]
Sep 10 15:17:03.820671 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.820641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02-nginx-conf\") pod \"networking-console-plugin-79977f9844-b2znc\" (UID: \"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:03.820813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.820675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b2znc\" (UID: \"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:03.846423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.846391    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74b4cd8b4d-45hnw"]
Sep 10 15:17:03.848017 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.848002    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.852725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.852705    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 15:17:03.853279 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.853263    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sxzrd\""
Sep 10 15:17:03.853707 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.853690    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 15:17:03.854335 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.854318    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 15:17:03.859948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.859928    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 15:17:03.870162 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.870138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:03.871580 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.871554    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b4cd8b4d-45hnw"]
Sep 10 15:17:03.921776 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.921741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33dca931-cc89-4713-b518-c93ef137e711-image-registry-private-configuration\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.921928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.921787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33dca931-cc89-4713-b518-c93ef137e711-installation-pull-secrets\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.921928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.921883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-bound-sa-token\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.921928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.921910    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc2xm\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-kube-api-access-jc2xm\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.922041 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.921990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33dca931-cc89-4713-b518-c93ef137e711-ca-trust-extracted\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.922041 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.922013    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33dca931-cc89-4713-b518-c93ef137e711-trusted-ca\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.922041 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.922032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02-nginx-conf\") pod \"networking-console-plugin-79977f9844-b2znc\" (UID: \"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:03.922145 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.922054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b2znc\" (UID: \"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:03.922145 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.922071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-registry-tls\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.922145 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.922087    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33dca931-cc89-4713-b518-c93ef137e711-registry-certificates\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:03.922702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.922685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02-nginx-conf\") pod \"networking-console-plugin-79977f9844-b2znc\" (UID: \"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:03.924300 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.924284    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-b2znc\" (UID: \"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02\") " pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:03.950609 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:03.950585    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:04.023148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33dca931-cc89-4713-b518-c93ef137e711-trusted-ca\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-registry-tls\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33dca931-cc89-4713-b518-c93ef137e711-registry-certificates\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33dca931-cc89-4713-b518-c93ef137e711-image-registry-private-configuration\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023426 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33dca931-cc89-4713-b518-c93ef137e711-installation-pull-secrets\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023426 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-bound-sa-token\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023426 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc2xm\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-kube-api-access-jc2xm\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023612 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33dca931-cc89-4713-b518-c93ef137e711-ca-trust-extracted\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.023891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33dca931-cc89-4713-b518-c93ef137e711-ca-trust-extracted\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.024047 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.023969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33dca931-cc89-4713-b518-c93ef137e711-registry-certificates\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.024330 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.024306    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33dca931-cc89-4713-b518-c93ef137e711-trusted-ca\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.025694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.025662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33dca931-cc89-4713-b518-c93ef137e711-image-registry-private-configuration\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.025694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.025691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-registry-tls\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.025806 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.025720    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33dca931-cc89-4713-b518-c93ef137e711-installation-pull-secrets\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.036118 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.036094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-bound-sa-token\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.036954 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.036938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc2xm\" (UniqueName: \"kubernetes.io/projected/33dca931-cc89-4713-b518-c93ef137e711-kube-api-access-jc2xm\") pod \"image-registry-74b4cd8b4d-45hnw\" (UID: \"33dca931-cc89-4713-b518-c93ef137e711\") " pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.066216 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.066192    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc"
Sep 10 15:17:04.156941 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.156905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:04.203402 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.203373    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x5bkx"
Sep 10 15:17:04.330578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.330465    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-b2znc"]
Sep 10 15:17:04.332855 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:04.332834    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b5b7d9_0c2a_4c3a_a461_d8a4414b3a02.slice/crio-270cfb8c635d7018f7da09504b8f6e81415efe9b399bc59bb4710bd52e225bbd WatchSource:0}: Error finding container 270cfb8c635d7018f7da09504b8f6e81415efe9b399bc59bb4710bd52e225bbd: Status 404 returned error can't find the container with id 270cfb8c635d7018f7da09504b8f6e81415efe9b399bc59bb4710bd52e225bbd
Sep 10 15:17:04.423228 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.423202    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b4cd8b4d-45hnw"]
Sep 10 15:17:04.426525 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:04.426505    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33dca931_cc89_4713_b518_c93ef137e711.slice/crio-d97e9026854f31457ea91a09e866ab3623dcd824c3fdc61dc506bf5cbb7e1254 WatchSource:0}: Error finding container d97e9026854f31457ea91a09e866ab3623dcd824c3fdc61dc506bf5cbb7e1254: Status 404 returned error can't find the container with id d97e9026854f31457ea91a09e866ab3623dcd824c3fdc61dc506bf5cbb7e1254
Sep 10 15:17:04.585638 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.585557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:04.684594 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.684563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:04.756742 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.756708    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"]
Sep 10 15:17:04.758572 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.758543    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"
Sep 10 15:17:04.760131 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.760108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:04.760984 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.760966    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 15:17:04.760984 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.760981    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bjkzq\""
Sep 10 15:17:04.773087 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.773065    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"]
Sep 10 15:17:04.828865 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.828829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fde0e148-ef46-41bc-a503-0f7770774b1f-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-n2fs8\" (UID: \"fde0e148-ef46-41bc-a503-0f7770774b1f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"
Sep 10 15:17:04.847161 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.847084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:04.929774 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.929747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fde0e148-ef46-41bc-a503-0f7770774b1f-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-n2fs8\" (UID: \"fde0e148-ef46-41bc-a503-0f7770774b1f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"
Sep 10 15:17:04.932160 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:04.932134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fde0e148-ef46-41bc-a503-0f7770774b1f-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-n2fs8\" (UID: \"fde0e148-ef46-41bc-a503-0f7770774b1f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"
Sep 10 15:17:05.070171 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.070132    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"
Sep 10 15:17:05.231561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.231525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw" event={"ID":"33dca931-cc89-4713-b518-c93ef137e711","Type":"ContainerStarted","Data":"63b169d371f674920052039a73d06901e2541b49b968ef67b00d96d22d7d5f1c"}
Sep 10 15:17:05.231702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.231574    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw" event={"ID":"33dca931-cc89-4713-b518-c93ef137e711","Type":"ContainerStarted","Data":"d97e9026854f31457ea91a09e866ab3623dcd824c3fdc61dc506bf5cbb7e1254"}
Sep 10 15:17:05.231702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.231609    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:05.232677 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.232657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc" event={"ID":"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02","Type":"ContainerStarted","Data":"270cfb8c635d7018f7da09504b8f6e81415efe9b399bc59bb4710bd52e225bbd"}
Sep 10 15:17:05.251920 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.251877    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw" podStartSLOduration=2.251863808 podStartE2EDuration="2.251863808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:05.251077339 +0000 UTC m=+65.787032918" watchObservedRunningTime="2026-09-10 15:17:05.251863808 +0000 UTC m=+65.787819381"
Sep 10 15:17:05.335588 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.335561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"]
Sep 10 15:17:05.338328 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:05.338305    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde0e148_ef46_41bc_a503_0f7770774b1f.slice/crio-dd026e0e864534a27bfb8d351dec4b05f8088f78e9f0794c5566d3882e6a90d5 WatchSource:0}: Error finding container dd026e0e864534a27bfb8d351dec4b05f8088f78e9f0794c5566d3882e6a90d5: Status 404 returned error can't find the container with id dd026e0e864534a27bfb8d351dec4b05f8088f78e9f0794c5566d3882e6a90d5
Sep 10 15:17:05.634683 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.634643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:17:05.637748 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.637721    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 15:17:05.647795 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.647771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a454b8ce-8961-450d-ada9-bdb7489af486-metrics-certs\") pod \"network-metrics-daemon-t8jf6\" (UID: \"a454b8ce-8961-450d-ada9-bdb7489af486\") " pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:17:05.735712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.735671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:17:05.735897 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.735745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:17:05.738816 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.738787    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 15:17:05.738921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.738818    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 15:17:05.748392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.748371    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 15:17:05.748532 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.748465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75f17da5-a2cb-460c-b5ad-5804f13505dc-original-pull-secret\") pod \"global-pull-secret-syncer-z98zv\" (UID: \"75f17da5-a2cb-460c-b5ad-5804f13505dc\") " pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:17:05.759374 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.759344    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjggj\" (UniqueName: \"kubernetes.io/projected/8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d-kube-api-access-wjggj\") pod \"network-check-target-dhxd7\" (UID: \"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d\") " pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:17:05.763666 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.763643    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z98zv"
Sep 10 15:17:05.771438 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.771408    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5wmh\""
Sep 10 15:17:05.778500 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:05.778476    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t8jf6"
Sep 10 15:17:06.068436 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.068417    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-96bxt\""
Sep 10 15:17:06.069536 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.069416    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:17:06.236933 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.236902    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8" event={"ID":"fde0e148-ef46-41bc-a503-0f7770774b1f","Type":"ContainerStarted","Data":"dd026e0e864534a27bfb8d351dec4b05f8088f78e9f0794c5566d3882e6a90d5"}
Sep 10 15:17:06.277913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.277834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:06.278036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.277924    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z98zv"]
Sep 10 15:17:06.280770 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.280746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t8jf6"]
Sep 10 15:17:06.351830 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.351796    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dhxd7"]
Sep 10 15:17:06.355572 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:06.355546    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:06.382815 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:06.382785    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75f17da5_a2cb_460c_b5ad_5804f13505dc.slice/crio-7adc537c0e7051f9b0c3d34391cb052f45dfad9c0cbaf7c98f7be85e4d3849d6 WatchSource:0}: Error finding container 7adc537c0e7051f9b0c3d34391cb052f45dfad9c0cbaf7c98f7be85e4d3849d6: Status 404 returned error can't find the container with id 7adc537c0e7051f9b0c3d34391cb052f45dfad9c0cbaf7c98f7be85e4d3849d6
Sep 10 15:17:06.395557 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:06.395530    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8acb78d6_4885_47b0_b03b_1a6c9d3c9e7d.slice/crio-7737ee42e76dffd373977519e23bdb10e94c4504bc17f355eddbad0aa4d6b3ca WatchSource:0}: Error finding container 7737ee42e76dffd373977519e23bdb10e94c4504bc17f355eddbad0aa4d6b3ca: Status 404 returned error can't find the container with id 7737ee42e76dffd373977519e23bdb10e94c4504bc17f355eddbad0aa4d6b3ca
Sep 10 15:17:07.242495 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.242437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc" event={"ID":"61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02","Type":"ContainerStarted","Data":"48e03b5de5e9a6684abe7a56c78eaf00558266f06480fb57779df2e6cf9c1af3"}
Sep 10 15:17:07.243817 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.243787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t8jf6" event={"ID":"a454b8ce-8961-450d-ada9-bdb7489af486","Type":"ContainerStarted","Data":"3fe3fa4dbe4871335612263301bab0c5382285df31143791588d2bf537bf6205"}
Sep 10 15:17:07.245086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.245060    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z98zv" event={"ID":"75f17da5-a2cb-460c-b5ad-5804f13505dc","Type":"ContainerStarted","Data":"7adc537c0e7051f9b0c3d34391cb052f45dfad9c0cbaf7c98f7be85e4d3849d6"}
Sep 10 15:17:07.246264 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.246239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dhxd7" event={"ID":"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d","Type":"ContainerStarted","Data":"7737ee42e76dffd373977519e23bdb10e94c4504bc17f355eddbad0aa4d6b3ca"}
Sep 10 15:17:07.264995 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.264951    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-b2znc" podStartSLOduration=2.185909153 podStartE2EDuration="4.264936722s" totalImagesPullingTime="2.079027569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:07.264331499 +0000 UTC m=+67.800287060" watchObservedRunningTime="2026-09-10 15:17:07.264936722 +0000 UTC m=+67.800892646"
Sep 10 15:17:07.367895 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.367837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:07.469232 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.469200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:07.667601 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.667530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:07.735042 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:07.735009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:08.252397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:08.252359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8" event={"ID":"fde0e148-ef46-41bc-a503-0f7770774b1f","Type":"ContainerStarted","Data":"36b1ae9aa9ade249ff1e516babeedf325d2a5ccd3dc5e8ef3faade3887df4635"}
Sep 10 15:17:08.271083 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:08.271033    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8" podStartSLOduration=2.247548456 podStartE2EDuration="4.271013535s" totalImagesPullingTime="2.023465079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:08.270882336 +0000 UTC m=+68.806837916" watchObservedRunningTime="2026-09-10 15:17:08.271013535 +0000 UTC m=+68.806969098"
Sep 10 15:17:09.260528 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.260442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t8jf6" event={"ID":"a454b8ce-8961-450d-ada9-bdb7489af486","Type":"ContainerStarted","Data":"0fd8f4ebb80dacd5fddf99953acdd3a6f398cae0b85790e60ed349fba5db67e9"}
Sep 10 15:17:09.260979 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.260859    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"
Sep 10 15:17:09.261413 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.261384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t8jf6" event={"ID":"a454b8ce-8961-450d-ada9-bdb7489af486","Type":"ContainerStarted","Data":"f7d5dc927e592ad96ee00cb41c469e4e5e76e17a244ea319242702d85979aff1"}
Sep 10 15:17:09.266635 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.266608    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-n2fs8"
Sep 10 15:17:09.280016 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.279970    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t8jf6" podStartSLOduration=67.403688495 podStartE2EDuration="1m9.279956742s" totalImagesPullingTime="1.876268247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:09.278649686 +0000 UTC m=+69.814605273" watchObservedRunningTime="2026-09-10 15:17:09.279956742 +0000 UTC m=+69.815912305"
Sep 10 15:17:09.383873 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.383800    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:09.485303 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.485252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:09.586682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.586647    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:09.784614 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.784579    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:09.865712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.865628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:09.925523 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:09.925487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:10.172561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:10.172476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:17:10.175129 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:10.175102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7182f42f-a1d6-405e-bf4d-f6ffee314b6d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8sqc\" (UID: \"7182f42f-a1d6-405e-bf4d-f6ffee314b6d\") " pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:17:10.444047 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:10.443970    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-696sd\""
Sep 10 15:17:10.451995 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:10.451965    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8sqc"
Sep 10 15:17:11.588632 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:11.588550    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8sqc"]
Sep 10 15:17:11.591776 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:11.591745    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7182f42f_a1d6_405e_bf4d_f6ffee314b6d.slice/crio-48fa4be4f82cc559239dc646e0625b4d0469dc31cfeabbd54977cb50643ce8fe WatchSource:0}: Error finding container 48fa4be4f82cc559239dc646e0625b4d0469dc31cfeabbd54977cb50643ce8fe: Status 404 returned error can't find the container with id 48fa4be4f82cc559239dc646e0625b4d0469dc31cfeabbd54977cb50643ce8fe
Sep 10 15:17:12.187972 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.187941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:12.274462 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.274414    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8sqc" event={"ID":"7182f42f-a1d6-405e-bf4d-f6ffee314b6d","Type":"ContainerStarted","Data":"1375f36c3041e4b174144ba8485b73856222421d01aaddc135b47f97cc80c0c2"}
Sep 10 15:17:12.274624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.274490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8sqc" event={"ID":"7182f42f-a1d6-405e-bf4d-f6ffee314b6d","Type":"ContainerStarted","Data":"48fa4be4f82cc559239dc646e0625b4d0469dc31cfeabbd54977cb50643ce8fe"}
Sep 10 15:17:12.276365 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.276329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z98zv" event={"ID":"75f17da5-a2cb-460c-b5ad-5804f13505dc","Type":"ContainerStarted","Data":"1921902c5677e5cd8d11bd9c9bfd000b2941a115f52504c4db3d7c10e8e6324e"}
Sep 10 15:17:12.278140 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.278108    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dhxd7" event={"ID":"8acb78d6-4885-47b0-b03b-1a6c9d3c9e7d","Type":"ContainerStarted","Data":"d93f8e3f4d3c4f4fc735217b0cc1273b7afe7018aa5a7f5674bcd30f2c1d72a6"}
Sep 10 15:17:12.278274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.278258    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:17:12.302051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.302006    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z98zv" podStartSLOduration=66.355069292 podStartE2EDuration="1m11.30199382s" totalImagesPullingTime="4.946924528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:12.30177872 +0000 UTC m=+72.837734286" watchObservedRunningTime="2026-09-10 15:17:12.30199382 +0000 UTC m=+72.837949387"
Sep 10 15:17:12.326050 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:12.325994    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dhxd7" podStartSLOduration=67.41177357 podStartE2EDuration="1m12.325979648s" totalImagesPullingTime="4.914206078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:12.324398426 +0000 UTC m=+72.860353991" watchObservedRunningTime="2026-09-10 15:17:12.325979648 +0000 UTC m=+72.861935211"
Sep 10 15:17:13.283491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:13.283436    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8sqc" event={"ID":"7182f42f-a1d6-405e-bf4d-f6ffee314b6d","Type":"ContainerStarted","Data":"64712085cf0df11aa05c80b35fae71f08641abd66bfb0847b4bd13c08ad118b8"}
Sep 10 15:17:13.358569 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:13.358533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:13.457549 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:13.457514    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:14.289015 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:14.288983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8sqc" event={"ID":"7182f42f-a1d6-405e-bf4d-f6ffee314b6d","Type":"ContainerStarted","Data":"a6ca46a979d57e6e2d1d61d01ea82dee7a2dfdb98576d760d267b73bf6b9b17c"}
Sep 10 15:17:14.310343 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:14.310300    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p8sqc" podStartSLOduration=33.928150698 podStartE2EDuration="36.310285876s" totalImagesPullingTime="2.382135178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:16:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:14.309986146 +0000 UTC m=+74.845941727" watchObservedRunningTime="2026-09-10 15:17:14.310285876 +0000 UTC m=+74.846241438"
Sep 10 15:17:14.917059 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:14.917027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.029766 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.029731    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.142160 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.142124    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.168277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.168203    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"]
Sep 10 15:17:15.170488 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.170470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.173906 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.173886    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 15:17:15.174265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.173886    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 15:17:15.174265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.173899    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2tbkg\""
Sep 10 15:17:15.174265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.174092    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 15:17:15.174265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.173924    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 15:17:15.174265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.173937    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 15:17:15.182240 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.182216    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"]
Sep 10 15:17:15.193107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.193085    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"]
Sep 10 15:17:15.195307 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.195291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.198178 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.198162    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hdz7d\""
Sep 10 15:17:15.198275 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.198195    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 15:17:15.198613 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.198591    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 15:17:15.198613 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.198619    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 15:17:15.207313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.207291    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gwjdd"]
Sep 10 15:17:15.209428 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.209411    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.211726 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.211709    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m2csv\""
Sep 10 15:17:15.211841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.211730    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 15:17:15.212423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.212283    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 15:17:15.212710 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.212626    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 15:17:15.214320 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.214299    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"]
Sep 10 15:17:15.226890 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.226868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.301670 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.301616    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.307146 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.307271 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-root\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307271 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-wtmp\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307271 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m7ls\" (UniqueName: \"kubernetes.io/projected/da91d9d1-45cd-479b-be14-96dace47f489-kube-api-access-2m7ls\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307271 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307212    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-textfile\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27b8e716-2ad5-429d-b692-acdceb3d02bb-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.307431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9vcd\" (UniqueName: \"kubernetes.io/projected/27b8e716-2ad5-429d-b692-acdceb3d02bb-kube-api-access-g9vcd\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.307431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/68d4721f-7567-4739-ac9f-5a2db94bf08e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.307431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307397    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqfsg\" (UniqueName: \"kubernetes.io/projected/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-api-access-sqfsg\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.307431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307425    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.307689 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307689 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.307689 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.307689 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68d4721f-7567-4739-ac9f-5a2db94bf08e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.307689 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307650    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307691    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.307891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-tls\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da91d9d1-45cd-479b-be14-96dace47f489-metrics-client-ca\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.307891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.307796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-sys\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.403100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.403065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.408737 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-textfile\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.408862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408750    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27b8e716-2ad5-429d-b692-acdceb3d02bb-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.408862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9vcd\" (UniqueName: \"kubernetes.io/projected/27b8e716-2ad5-429d-b692-acdceb3d02bb-kube-api-access-g9vcd\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.408862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/68d4721f-7567-4739-ac9f-5a2db94bf08e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.408862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqfsg\" (UniqueName: \"kubernetes.io/projected/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-api-access-sqfsg\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.409066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.409066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.409066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.409066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.408988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68d4721f-7567-4739-ac9f-5a2db94bf08e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.409066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409063    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-tls\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da91d9d1-45cd-479b-be14-96dace47f489-metrics-client-ca\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409147    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-sys\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-textfile\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-root\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:15.409298    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-wtmp\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m7ls\" (UniqueName: \"kubernetes.io/projected/da91d9d1-45cd-479b-be14-96dace47f489-kube-api-access-2m7ls\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409392 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:15.409359    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-tls podName:27b8e716-2ad5-429d-b692-acdceb3d02bb nodeName:}" failed. No retries permitted until 2026-09-10 15:17:15.909338858 +0000 UTC m=+76.445294403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-gchdd" (UID: "27b8e716-2ad5-429d-b692-acdceb3d02bb") : secret "openshift-state-metrics-tls" not found
Sep 10 15:17:15.409879 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-root\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.409934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.409905    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-wtmp\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.410040 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:15.410011    2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 15:17:15.410102 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.410077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27b8e716-2ad5-429d-b692-acdceb3d02bb-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.410804 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.410779    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68d4721f-7567-4739-ac9f-5a2db94bf08e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.410903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.410818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.410903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.410847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da91d9d1-45cd-479b-be14-96dace47f489-sys\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.411177 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:15.410084    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-tls podName:68d4721f-7567-4739-ac9f-5a2db94bf08e nodeName:}" failed. No retries permitted until 2026-09-10 15:17:15.91006445 +0000 UTC m=+76.446020011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-cbnsb" (UID: "68d4721f-7567-4739-ac9f-5a2db94bf08e") : secret "kube-state-metrics-tls" not found
Sep 10 15:17:15.411683 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.411660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/68d4721f-7567-4739-ac9f-5a2db94bf08e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.412380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.411787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.412512 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.412028    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da91d9d1-45cd-479b-be14-96dace47f489-metrics-client-ca\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.414157 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.414129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.416527 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.416125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da91d9d1-45cd-479b-be14-96dace47f489-node-exporter-tls\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.416633 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.416548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.416990 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.416969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.419253 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.419193    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m7ls\" (UniqueName: \"kubernetes.io/projected/da91d9d1-45cd-479b-be14-96dace47f489-kube-api-access-2m7ls\") pod \"node-exporter-gwjdd\" (UID: \"da91d9d1-45cd-479b-be14-96dace47f489\") " pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.423600 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.423072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9vcd\" (UniqueName: \"kubernetes.io/projected/27b8e716-2ad5-429d-b692-acdceb3d02bb-kube-api-access-g9vcd\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.423600 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.423566    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqfsg\" (UniqueName: \"kubernetes.io/projected/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-api-access-sqfsg\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.503382 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.503346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.520326 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.520295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gwjdd"
Sep 10 15:17:15.530140 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:15.530111    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda91d9d1_45cd_479b_be14_96dace47f489.slice/crio-570cdae6f52836dba4fef3a8254d7fd4e7cd011fc44953b3ce34be8c03df49b2 WatchSource:0}: Error finding container 570cdae6f52836dba4fef3a8254d7fd4e7cd011fc44953b3ce34be8c03df49b2: Status 404 returned error can't find the container with id 570cdae6f52836dba4fef3a8254d7fd4e7cd011fc44953b3ce34be8c03df49b2
Sep 10 15:17:15.608799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.608765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.709429 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.709344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.762863 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.762832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:15.914223 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.914188    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:15.914399 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.914254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.916555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.916533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b8e716-2ad5-429d-b692-acdceb3d02bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-gchdd\" (UID: \"27b8e716-2ad5-429d-b692-acdceb3d02bb\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:15.916637 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:15.916563    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68d4721f-7567-4739-ac9f-5a2db94bf08e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-cbnsb\" (UID: \"68d4721f-7567-4739-ac9f-5a2db94bf08e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:16.081253 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.081215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"
Sep 10 15:17:16.107315 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.105242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"
Sep 10 15:17:16.142712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.142633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:16.223082 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.223050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:16.236139 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.236110    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:17:16.240435 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.240414    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.243091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.243026    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 15:17:16.243091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.243089    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 15:17:16.243478 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.243035    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 15:17:16.243478 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.243286    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 15:17:16.243478 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.243351    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 15:17:16.243478 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.243398    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 15:17:16.244278 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.244251    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 15:17:16.244386 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.244367    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9ntxh\""
Sep 10 15:17:16.244592 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.244573    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 15:17:16.244660 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.244632    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 15:17:16.253586 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.253563    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:17:16.299299 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.299268    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwjdd" event={"ID":"da91d9d1-45cd-479b-be14-96dace47f489","Type":"ContainerStarted","Data":"570cdae6f52836dba4fef3a8254d7fd4e7cd011fc44953b3ce34be8c03df49b2"}
Sep 10 15:17:16.317700 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317935    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-volume\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.317962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.318046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp5xc\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-kube-api-access-vp5xc\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.318088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-web-config\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318113 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.318116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.318626 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.318150    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-out\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.365099 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.365072    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd"]
Sep 10 15:17:16.405805 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.405773    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb"]
Sep 10 15:17:16.418794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.418770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.418921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.418808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.418921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.418850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-volume\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.418921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.418875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419077 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.418940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp5xc\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-kube-api-access-vp5xc\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419077 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.418985    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-web-config\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419077 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419077 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419060    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-out\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419129    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419156    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.419542 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:16.419365    2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 10 15:17:16.419542 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:16.419427    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls podName:ecb4b823-69ba-4acb-a114-750d4dcf318d nodeName:}" failed. No retries permitted until 2026-09-10 15:17:16.919407265 +0000 UTC m=+77.455362873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d") : secret "alertmanager-main-tls" not found
Sep 10 15:17:16.419661 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.419588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.420482 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.420407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.420673 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.420607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.422203 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.422175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-volume\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.422587 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.422545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.422932 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.422909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-web-config\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.423280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.423254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.423361 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.423288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.424029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.424007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.424286 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.424252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-out\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.425024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.425001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.427651 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.427632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp5xc\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-kube-api-access-vp5xc\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.542651 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.542618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:16.554435 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:16.554397    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b8e716_2ad5_429d_b692_acdceb3d02bb.slice/crio-9c476a81e148f85a591128dd46b7723fe060693cce5ead0cbed28f02f0ad7f6c WatchSource:0}: Error finding container 9c476a81e148f85a591128dd46b7723fe060693cce5ead0cbed28f02f0ad7f6c: Status 404 returned error can't find the container with id 9c476a81e148f85a591128dd46b7723fe060693cce5ead0cbed28f02f0ad7f6c
Sep 10 15:17:16.555120 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:16.555102    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d4721f_7567_4739_ac9f_5a2db94bf08e.slice/crio-aa77170cdbdf9e92e710a333a2ed2180f55258af0a16ac976a4a8c6a16c13874 WatchSource:0}: Error finding container aa77170cdbdf9e92e710a333a2ed2180f55258af0a16ac976a4a8c6a16c13874: Status 404 returned error can't find the container with id aa77170cdbdf9e92e710a333a2ed2180f55258af0a16ac976a4a8c6a16c13874
Sep 10 15:17:16.746351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.746316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:16.839814 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.839786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:16.924283 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.924198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:16.926533 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:16.926510    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:17.152938 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.152903    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:17:17.304843 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.304801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb" event={"ID":"68d4721f-7567-4739-ac9f-5a2db94bf08e","Type":"ContainerStarted","Data":"aa77170cdbdf9e92e710a333a2ed2180f55258af0a16ac976a4a8c6a16c13874"}
Sep 10 15:17:17.307189 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.306813    2562 generic.go:356] "Generic (PLEG): container finished" podID="da91d9d1-45cd-479b-be14-96dace47f489" containerID="4cf4047700d9b871a863be485b0e193dec3ae3ae5fb1bc3b418a1c618b444f3a" exitCode=0
Sep 10 15:17:17.307189 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.306849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwjdd" event={"ID":"da91d9d1-45cd-479b-be14-96dace47f489","Type":"ContainerDied","Data":"4cf4047700d9b871a863be485b0e193dec3ae3ae5fb1bc3b418a1c618b444f3a"}
Sep 10 15:17:17.309653 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.309626    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd" event={"ID":"27b8e716-2ad5-429d-b692-acdceb3d02bb","Type":"ContainerStarted","Data":"027cb49f7849df9aa3005c3b198ed807ee5619f0eba05f401f5df841ab2f6e9d"}
Sep 10 15:17:17.309768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.309677    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd" event={"ID":"27b8e716-2ad5-429d-b692-acdceb3d02bb","Type":"ContainerStarted","Data":"d890f050735e9cee7d2655b2fcaa9b0c64f43f5e58875c54e0106a12b7998daf"}
Sep 10 15:17:17.309768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.309736    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd" event={"ID":"27b8e716-2ad5-429d-b692-acdceb3d02bb","Type":"ContainerStarted","Data":"9c476a81e148f85a591128dd46b7723fe060693cce5ead0cbed28f02f0ad7f6c"}
Sep 10 15:17:17.442218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.442044    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:17:17.446349 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:17.446319    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb4b823_69ba_4acb_a114_750d4dcf318d.slice/crio-e1fde61f89ac9123e7c94c3ed120bbcdcf70ca0a0238f83706c4ae5c4a5de353 WatchSource:0}: Error finding container e1fde61f89ac9123e7c94c3ed120bbcdcf70ca0a0238f83706c4ae5c4a5de353: Status 404 returned error can't find the container with id e1fde61f89ac9123e7c94c3ed120bbcdcf70ca0a0238f83706c4ae5c4a5de353
Sep 10 15:17:17.457221 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:17.457055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:18.183556 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.183529    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-55865655fd-78bnb"]
Sep 10 15:17:18.187014 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.186997    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.189401 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.189384    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7hm4h\""
Sep 10 15:17:18.189545 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.189431    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 15:17:18.190193 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.190174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d9nrspbnl5pr2\""
Sep 10 15:17:18.190193 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.190188    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 15:17:18.190645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.190444    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 15:17:18.190645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.190598    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 15:17:18.193640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.193617    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 15:17:18.205936 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.205909    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55865655fd-78bnb"]
Sep 10 15:17:18.323630 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.323599    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb" event={"ID":"68d4721f-7567-4739-ac9f-5a2db94bf08e","Type":"ContainerStarted","Data":"e7560ae9262c83331eb07a5334e8cd96bdc97a94946e8352b4e9fb853a364e98"}
Sep 10 15:17:18.323810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.323789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb" event={"ID":"68d4721f-7567-4739-ac9f-5a2db94bf08e","Type":"ContainerStarted","Data":"e131a331a7461cffa19928c996b2fae711bf1cf4406fc6460debeeb338e94367"}
Sep 10 15:17:18.328781 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.328743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwjdd" event={"ID":"da91d9d1-45cd-479b-be14-96dace47f489","Type":"ContainerStarted","Data":"03b7e6b2f67207efa1bf36bec3ff362bfb5d5a573269f6da00e0dac5a33e3df6"}
Sep 10 15:17:18.328887 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.328803    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwjdd" event={"ID":"da91d9d1-45cd-479b-be14-96dace47f489","Type":"ContainerStarted","Data":"0e4cdbedbb625e9b3d15ff248d17b29ed55f658a0957a5d07c50c24d65b70e2e"}
Sep 10 15:17:18.338944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.335869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd" event={"ID":"27b8e716-2ad5-429d-b692-acdceb3d02bb","Type":"ContainerStarted","Data":"c637d51b55da01fbe62c45d78675b0ed516517c0337434244b7441bf1181c640"}
Sep 10 15:17:18.338944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.338814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csbfg\" (UniqueName: \"kubernetes.io/projected/5726236c-0207-45cf-b08f-bc28f20796ce-kube-api-access-csbfg\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.338944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.338914    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5726236c-0207-45cf-b08f-bc28f20796ce-metrics-client-ca\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.338944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.338943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-grpc-tls\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.339284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.338982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.339284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.339023    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.339284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.339074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.339284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.339102    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-tls\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.339284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.339149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.341031 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.341010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerStarted","Data":"e1fde61f89ac9123e7c94c3ed120bbcdcf70ca0a0238f83706c4ae5c4a5de353"}
Sep 10 15:17:18.349056 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.349026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:18.369586 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.369025    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gwjdd" podStartSLOduration=2.240081379 podStartE2EDuration="3.369003243s" totalImagesPullingTime="1.066046024s" totalInitContainerRuntime="62.87584ms" isStatefulPod=true podCreationTimestamp="2026-09-10 15:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:18.367824947 +0000 UTC m=+78.903780539" watchObservedRunningTime="2026-09-10 15:17:18.369003243 +0000 UTC m=+78.904958806"
Sep 10 15:17:18.389149 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.389091    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-gchdd" podStartSLOduration=1.9284538709999999 podStartE2EDuration="3.389070683s" totalImagesPullingTime="1.460616812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:18.388711299 +0000 UTC m=+78.924666886" watchObservedRunningTime="2026-09-10 15:17:18.389070683 +0000 UTC m=+78.925026247"
Sep 10 15:17:18.435174 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.435134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:18.440364 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440321    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.440555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csbfg\" (UniqueName: \"kubernetes.io/projected/5726236c-0207-45cf-b08f-bc28f20796ce-kube-api-access-csbfg\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.440555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5726236c-0207-45cf-b08f-bc28f20796ce-metrics-client-ca\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.440555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-grpc-tls\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.440555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.440802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.440802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.440802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.440718    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-tls\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.442387 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.442346    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5726236c-0207-45cf-b08f-bc28f20796ce-metrics-client-ca\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.445016 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.444944    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.445247 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.445223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.445608 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.445571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-tls\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.446442 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.446397    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.446757 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.446689    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-grpc-tls\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.447931 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.447890    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5726236c-0207-45cf-b08f-bc28f20796ce-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.456515 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.456435    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csbfg\" (UniqueName: \"kubernetes.io/projected/5726236c-0207-45cf-b08f-bc28f20796ce-kube-api-access-csbfg\") pod \"thanos-querier-55865655fd-78bnb\" (UID: \"5726236c-0207-45cf-b08f-bc28f20796ce\") " pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.498586 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.498547    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:18.636333 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.636254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:18.798975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.798769    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55865655fd-78bnb"]
Sep 10 15:17:18.880381 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:18.880344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:18.918617 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:18.918548    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5726236c_0207_45cf_b08f_bc28f20796ce.slice/crio-b634447f455b779a6a8793cf0db5142150bf66bb22842166c29a5ffbfbdcdab4 WatchSource:0}: Error finding container b634447f455b779a6a8793cf0db5142150bf66bb22842166c29a5ffbfbdcdab4: Status 404 returned error can't find the container with id b634447f455b779a6a8793cf0db5142150bf66bb22842166c29a5ffbfbdcdab4
Sep 10 15:17:19.346682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.346640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb" event={"ID":"68d4721f-7567-4739-ac9f-5a2db94bf08e","Type":"ContainerStarted","Data":"bf1dd0425eb764b910421ba8d1c944397db3e6745f41e5c69de77d798423ac41"}
Sep 10 15:17:19.347836 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.347806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" event={"ID":"5726236c-0207-45cf-b08f-bc28f20796ce","Type":"ContainerStarted","Data":"b634447f455b779a6a8793cf0db5142150bf66bb22842166c29a5ffbfbdcdab4"}
Sep 10 15:17:19.349184 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.349161    2562 generic.go:356] "Generic (PLEG): container finished" podID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerID="811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396" exitCode=0
Sep 10 15:17:19.349259 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.349236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396"}
Sep 10 15:17:19.368323 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.368278    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cbnsb" podStartSLOduration=2.80464846 podStartE2EDuration="4.368263594s" totalImagesPullingTime="1.563615134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:19.366884532 +0000 UTC m=+79.902840117" watchObservedRunningTime="2026-09-10 15:17:19.368263594 +0000 UTC m=+79.904219186"
Sep 10 15:17:19.479340 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.479298    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-559bd765d7-9fdj2"]
Sep 10 15:17:19.482737 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.482718    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.485751 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.485728    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2h79efn93tv7k\""
Sep 10 15:17:19.485751 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.485740    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 15:17:19.485920 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.485911    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 15:17:19.485973 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.485948    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 15:17:19.486030 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.485969    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gtpw8\""
Sep 10 15:17:19.486129 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.486102    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 15:17:19.493267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.493243    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-559bd765d7-9fdj2"]
Sep 10 15:17:19.549229 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.549195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh9vr\" (UniqueName: \"kubernetes.io/projected/08e15dc8-5946-4658-9704-d775937e989d-kube-api-access-nh9vr\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.549229 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.549236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/08e15dc8-5946-4658-9704-d775937e989d-metrics-server-audit-profiles\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.549504 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.549275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-secret-metrics-server-tls\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.549504 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.549337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/08e15dc8-5946-4658-9704-d775937e989d-audit-log\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.549504 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.549367    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-client-ca-bundle\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.549504 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.549465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-secret-metrics-server-client-certs\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.549712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.549511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e15dc8-5946-4658-9704-d775937e989d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650164 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e15dc8-5946-4658-9704-d775937e989d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650164 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh9vr\" (UniqueName: \"kubernetes.io/projected/08e15dc8-5946-4658-9704-d775937e989d-kube-api-access-nh9vr\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650164 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/08e15dc8-5946-4658-9704-d775937e989d-metrics-server-audit-profiles\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-secret-metrics-server-tls\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/08e15dc8-5946-4658-9704-d775937e989d-audit-log\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-client-ca-bundle\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-secret-metrics-server-client-certs\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650663    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/08e15dc8-5946-4658-9704-d775937e989d-audit-log\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.650928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.650903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e15dc8-5946-4658-9704-d775937e989d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.651186 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.651167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/08e15dc8-5946-4658-9704-d775937e989d-metrics-server-audit-profiles\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.652856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.652831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-secret-metrics-server-client-certs\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.652938 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.652881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-secret-metrics-server-tls\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.652978 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.652943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/08e15dc8-5946-4658-9704-d775937e989d-client-ca-bundle\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.660353 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.660329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh9vr\" (UniqueName: \"kubernetes.io/projected/08e15dc8-5946-4658-9704-d775937e989d-kube-api-access-nh9vr\") pod \"metrics-server-559bd765d7-9fdj2\" (UID: \"08e15dc8-5946-4658-9704-d775937e989d\") " pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.795066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.795022    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:19.926617 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.926535    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:19.934413 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.934386    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"]
Sep 10 15:17:19.938155 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.938135    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:19.940881 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.940856    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 15:17:19.941342 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.941163    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qdz25\""
Sep 10 15:17:19.950725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:19.949685    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"]
Sep 10 15:17:20.054386 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.054268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwhmf\" (UniqueName: \"kubernetes.io/projected/ee6db9ec-40e9-4aa7-9452-43936d102961-kube-api-access-mwhmf\") pod \"monitoring-plugin-85bbbbbf4-kk4z4\" (UID: \"ee6db9ec-40e9-4aa7-9452-43936d102961\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:20.054386 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.054324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee6db9ec-40e9-4aa7-9452-43936d102961-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kk4z4\" (UID: \"ee6db9ec-40e9-4aa7-9452-43936d102961\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:20.072867 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.072844    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-559bd765d7-9fdj2"]
Sep 10 15:17:20.075257 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:20.075230    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e15dc8_5946_4658_9704_d775937e989d.slice/crio-8e97fe507eabaaf57054e5aba95bf21f6e04518078d9a28cd0b08351179268f1 WatchSource:0}: Error finding container 8e97fe507eabaaf57054e5aba95bf21f6e04518078d9a28cd0b08351179268f1: Status 404 returned error can't find the container with id 8e97fe507eabaaf57054e5aba95bf21f6e04518078d9a28cd0b08351179268f1
Sep 10 15:17:20.155632 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.155593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhmf\" (UniqueName: \"kubernetes.io/projected/ee6db9ec-40e9-4aa7-9452-43936d102961-kube-api-access-mwhmf\") pod \"monitoring-plugin-85bbbbbf4-kk4z4\" (UID: \"ee6db9ec-40e9-4aa7-9452-43936d102961\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:20.155786 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.155647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee6db9ec-40e9-4aa7-9452-43936d102961-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kk4z4\" (UID: \"ee6db9ec-40e9-4aa7-9452-43936d102961\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:20.155829 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:20.155790    2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 15:17:20.155864 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:20.155858    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6db9ec-40e9-4aa7-9452-43936d102961-monitoring-plugin-cert podName:ee6db9ec-40e9-4aa7-9452-43936d102961 nodeName:}" failed. No retries permitted until 2026-09-10 15:17:20.655826608 +0000 UTC m=+81.191782162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/ee6db9ec-40e9-4aa7-9452-43936d102961-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-kk4z4" (UID: "ee6db9ec-40e9-4aa7-9452-43936d102961") : secret "monitoring-plugin-cert" not found
Sep 10 15:17:20.167516 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.167490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwhmf\" (UniqueName: \"kubernetes.io/projected/ee6db9ec-40e9-4aa7-9452-43936d102961-kube-api-access-mwhmf\") pod \"monitoring-plugin-85bbbbbf4-kk4z4\" (UID: \"ee6db9ec-40e9-4aa7-9452-43936d102961\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:20.359969 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.359927    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2" event={"ID":"08e15dc8-5946-4658-9704-d775937e989d","Type":"ContainerStarted","Data":"8e97fe507eabaaf57054e5aba95bf21f6e04518078d9a28cd0b08351179268f1"}
Sep 10 15:17:20.395168 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.393698    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-56c7466dc5-spvlq"]
Sep 10 15:17:20.397754 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.397732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.401375 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.401352    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 15:17:20.402066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.401628    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7rhbg\""
Sep 10 15:17:20.402066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.401813    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 15:17:20.402066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.401878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 15:17:20.402066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.402018    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 15:17:20.402353 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.402330    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 15:17:20.408408 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.408348    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 15:17:20.410485 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.410463    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56c7466dc5-spvlq"]
Sep 10 15:17:20.438322 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.438288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:20.459415 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prgvq\" (UniqueName: \"kubernetes.io/projected/0b0c05d6-c031-46ff-a124-6914558773b6-kube-api-access-prgvq\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.459571 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-serving-certs-ca-bundle\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.459571 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-secret-telemeter-client\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.459571 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-metrics-client-ca\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.459571 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459537    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.459728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-telemeter-client-tls\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.459728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-federate-client-tls\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.459929 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.459907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.560854 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.560798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.560864    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prgvq\" (UniqueName: \"kubernetes.io/projected/0b0c05d6-c031-46ff-a124-6914558773b6-kube-api-access-prgvq\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.560895    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-serving-certs-ca-bundle\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.560948    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-secret-telemeter-client\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.560993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-metrics-client-ca\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.561059    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.561123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-telemeter-client-tls\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.561178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-federate-client-tls\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561847 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.561797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-metrics-client-ca\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.561973 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.561849    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-serving-certs-ca-bundle\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.562051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.562014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b0c05d6-c031-46ff-a124-6914558773b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.564247 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.564221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.564439 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.564415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-secret-telemeter-client\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.564545 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.564504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-telemeter-client-tls\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.564920 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.564879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0b0c05d6-c031-46ff-a124-6914558773b6-federate-client-tls\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.571629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.571602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prgvq\" (UniqueName: \"kubernetes.io/projected/0b0c05d6-c031-46ff-a124-6914558773b6-kube-api-access-prgvq\") pod \"telemeter-client-56c7466dc5-spvlq\" (UID: \"0b0c05d6-c031-46ff-a124-6914558773b6\") " pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.661803 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.661707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee6db9ec-40e9-4aa7-9452-43936d102961-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kk4z4\" (UID: \"ee6db9ec-40e9-4aa7-9452-43936d102961\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:20.664685 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.664648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee6db9ec-40e9-4aa7-9452-43936d102961-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kk4z4\" (UID: \"ee6db9ec-40e9-4aa7-9452-43936d102961\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:20.710360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.710324    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq"
Sep 10 15:17:20.853519 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:20.853470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:21.176591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.176482    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"]
Sep 10 15:17:21.180284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.180198    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56c7466dc5-spvlq"]
Sep 10 15:17:21.367219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.367187    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" event={"ID":"5726236c-0207-45cf-b08f-bc28f20796ce","Type":"ContainerStarted","Data":"9bab84b395293d31bfcb9bf451a8f4a49bf6dfac592ff9295f00d9d96ed585f4"}
Sep 10 15:17:21.426932 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:21.426899    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6db9ec_40e9_4aa7_9452_43936d102961.slice/crio-f853c661ebb1ab51d003478b86389472a7c47484a982e9c16bec9ee2435c3c2a WatchSource:0}: Error finding container f853c661ebb1ab51d003478b86389472a7c47484a982e9c16bec9ee2435c3c2a: Status 404 returned error can't find the container with id f853c661ebb1ab51d003478b86389472a7c47484a982e9c16bec9ee2435c3c2a
Sep 10 15:17:21.427965 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:21.427942    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0c05d6_c031_46ff_a124_6914558773b6.slice/crio-0d3c6a11b6cff0200360c3cfd4ffbdeb8d2a6adc56d0d5d0157abff23687589f WatchSource:0}: Error finding container 0d3c6a11b6cff0200360c3cfd4ffbdeb8d2a6adc56d0d5d0157abff23687589f: Status 404 returned error can't find the container with id 0d3c6a11b6cff0200360c3cfd4ffbdeb8d2a6adc56d0d5d0157abff23687589f
Sep 10 15:17:21.612594 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.612570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:21.618271 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.618218    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:17:21.622364 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.622348    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.624773 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.624582    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 15:17:21.624773 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.624680    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5wmhw\""
Sep 10 15:17:21.625557 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.625399    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 15:17:21.625557 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.625425    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 15:17:21.625557 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.625509    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 15:17:21.625827 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.625808    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 15:17:21.625949 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.625914    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 15:17:21.626659 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.626640    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cpi4rrse0opub\""
Sep 10 15:17:21.628177 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.628094    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 15:17:21.628338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.628301    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 15:17:21.628414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.628352    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 15:17:21.628414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.628378    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 15:17:21.628414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.628397    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 15:17:21.628846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.628734    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 15:17:21.628846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.628770    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 15:17:21.632090 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.632066    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 15:17:21.639640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.639615    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:17:21.774863 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.774813    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775039 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.774886    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775039 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.774919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775039 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.774990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775157    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4s2g\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-kube-api-access-r4s2g\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775217    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775335    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775566    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.775684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.775641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.867597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.867558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:21.876822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.876747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.876822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.876788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.876839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.876865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.876890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.876932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.876967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877055    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4s2g\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-kube-api-access-r4s2g\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877214    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877641 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.877641 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.877961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.878001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.878029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.878045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.878070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.878093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.878309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.878905 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.878688    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.879515 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.879167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.880377 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.880018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.881122 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.881090    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.881212 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.881140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.881212 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.881187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.881326 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.881298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.881633 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.881586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.882274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.882230    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.882392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.882370    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.883426 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.883405    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.883543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.883519    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.883726 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.883706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.883829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.883812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.890667 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.890633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4s2g\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-kube-api-access-r4s2g\") pod \"prometheus-k8s-0\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.933907 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.933876    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:21.970474 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:21.970427    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:22.304274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.304123    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:17:22.307422 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:17:22.307376    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd82cc40_4433_4a70_96d5_99c7d57ba35d.slice/crio-514942b8908ee2428e29c8cf3ac34572384323cc30da29867317bc0769ab304b WatchSource:0}: Error finding container 514942b8908ee2428e29c8cf3ac34572384323cc30da29867317bc0769ab304b: Status 404 returned error can't find the container with id 514942b8908ee2428e29c8cf3ac34572384323cc30da29867317bc0769ab304b
Sep 10 15:17:22.375702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.375656    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2" event={"ID":"08e15dc8-5946-4658-9704-d775937e989d","Type":"ContainerStarted","Data":"4e96d5f2e950aae0534465ea1d84ea4d9e79f279ed58be51130adfab2d5fa8c8"}
Sep 10 15:17:22.377328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.377302    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerStarted","Data":"514942b8908ee2428e29c8cf3ac34572384323cc30da29867317bc0769ab304b"}
Sep 10 15:17:22.385793 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.385734    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerStarted","Data":"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"}
Sep 10 15:17:22.385883 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.385810    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerStarted","Data":"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"}
Sep 10 15:17:22.385883 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.385852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerStarted","Data":"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"}
Sep 10 15:17:22.385947 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.385890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerStarted","Data":"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"}
Sep 10 15:17:22.385947 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.385930    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerStarted","Data":"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"}
Sep 10 15:17:22.387380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.387337    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq" event={"ID":"0b0c05d6-c031-46ff-a124-6914558773b6","Type":"ContainerStarted","Data":"0d3c6a11b6cff0200360c3cfd4ffbdeb8d2a6adc56d0d5d0157abff23687589f"}
Sep 10 15:17:22.389405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.389384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" event={"ID":"5726236c-0207-45cf-b08f-bc28f20796ce","Type":"ContainerStarted","Data":"76f52ae3c8f5628650f63dc1f5ee714ea0d04ece4e5d2e538907c498fc76fe36"}
Sep 10 15:17:22.389517 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.389465    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" event={"ID":"5726236c-0207-45cf-b08f-bc28f20796ce","Type":"ContainerStarted","Data":"30fcd773ffb26b5d81856be03b9effb344b1ca9a3c91306295e00ee94f9d4982"}
Sep 10 15:17:22.390668 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:22.390643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4" event={"ID":"ee6db9ec-40e9-4aa7-9452-43936d102961","Type":"ContainerStarted","Data":"f853c661ebb1ab51d003478b86389472a7c47484a982e9c16bec9ee2435c3c2a"}
Sep 10 15:17:23.398165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.398129    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88" exitCode=0
Sep 10 15:17:23.398659 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.398218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"}
Sep 10 15:17:23.402408 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.402376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerStarted","Data":"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"}
Sep 10 15:17:23.406094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.406069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" event={"ID":"5726236c-0207-45cf-b08f-bc28f20796ce","Type":"ContainerStarted","Data":"95cf96dd5c5cbf182284e1a4a9291edd94f9d31f0ea7cd50830f3572e39f2893"}
Sep 10 15:17:23.406180 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.406137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" event={"ID":"5726236c-0207-45cf-b08f-bc28f20796ce","Type":"ContainerStarted","Data":"425f40ca09aed1dd69da27ca31c563ea4ed4d6f36d4ba3d00df9d8a2843b0ab3"}
Sep 10 15:17:23.406233 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.406178    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" event={"ID":"5726236c-0207-45cf-b08f-bc28f20796ce","Type":"ContainerStarted","Data":"5623a87c39129c1e2412a6e7d34a94c55f4756132d656da66c834a93ea328a71"}
Sep 10 15:17:23.406233 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.406212    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:23.427590 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.427538    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2" podStartSLOduration=2.435014601 podStartE2EDuration="4.427517723s" totalImagesPullingTime="1.992503122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:22.398553744 +0000 UTC m=+82.934509309" watchObservedRunningTime="2026-09-10 15:17:23.427517723 +0000 UTC m=+83.963473284"
Sep 10 15:17:23.454829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.454765    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb" podStartSLOduration=2.421078219 podStartE2EDuration="5.454741045s" totalImagesPullingTime="3.033662826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:23.451435111 +0000 UTC m=+83.987390676" watchObservedRunningTime="2026-09-10 15:17:23.454741045 +0000 UTC m=+83.990696607"
Sep 10 15:17:23.480642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:23.480589    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.405582998 podStartE2EDuration="7.480569491s" totalImagesPullingTime="4.059619171s" totalInitContainerRuntime="15.367322ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:23.479148833 +0000 UTC m=+84.015104426" watchObservedRunningTime="2026-09-10 15:17:23.480569491 +0000 UTC m=+84.016525045"
Sep 10 15:17:24.082540 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:24.082498    2562 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b1311c2217d4fd7cb932b0ce7682ead73b65f6c0211a54241e7fd807135c5cd3"
Sep 10 15:17:24.082671 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:24.082614    2562 kuberuntime_manager.go:1459] "Unhandled Error" err="container prometheus start failed in pod prometheus-k8s-0_openshift-monitoring(cd82cc40-4433-4a70-96d5-99c7d57ba35d): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError"
Sep 10 15:17:24.378942 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:24.378907    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"prometheus\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d"
Sep 10 15:17:24.412928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.412892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerStarted","Data":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"}
Sep 10 15:17:24.413313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.412958    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerStarted","Data":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"}
Sep 10 15:17:24.413313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.413020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerStarted","Data":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"}
Sep 10 15:17:24.413313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.413050    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerStarted","Data":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"}
Sep 10 15:17:24.413313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.413091    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerStarted","Data":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"}
Sep 10 15:17:24.414562 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:24.414530    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"prometheus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b1311c2217d4fd7cb932b0ce7682ead73b65f6c0211a54241e7fd807135c5cd3\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d"
Sep 10 15:17:24.415286 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.415264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq" event={"ID":"0b0c05d6-c031-46ff-a124-6914558773b6","Type":"ContainerStarted","Data":"31fd1134208e8c8328d1f7cb334641fc4733008e1aeb81f03dd6eb19216f28d5"}
Sep 10 15:17:24.415374 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.415324    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq" event={"ID":"0b0c05d6-c031-46ff-a124-6914558773b6","Type":"ContainerStarted","Data":"ab37775d660d07511bc82a8e95c7e43c7872be2b94f1f7703bdea83976bbf1c7"}
Sep 10 15:17:24.415374 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.415355    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq" event={"ID":"0b0c05d6-c031-46ff-a124-6914558773b6","Type":"ContainerStarted","Data":"a9c0e9196ac3bd58ee6477790909bb0b3c075abc3eab7a31d4baac56b94c6944"}
Sep 10 15:17:24.416893 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.416866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4" event={"ID":"ee6db9ec-40e9-4aa7-9452-43936d102961","Type":"ContainerStarted","Data":"d96a80b161826ead5c24d99305438a817f93eeb5a4581cc7f0406f4d8a8c494a"}
Sep 10 15:17:24.417574 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.417556    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:24.421974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.421952    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4"
Sep 10 15:17:24.465884 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.465846    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kk4z4" podStartSLOduration=3.083550757 podStartE2EDuration="5.465832566s" totalImagesPullingTime="2.382281809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:24.465484764 +0000 UTC m=+85.001440546" watchObservedRunningTime="2026-09-10 15:17:24.465832566 +0000 UTC m=+85.001788168"
Sep 10 15:17:24.489245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:24.489195    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-56c7466dc5-spvlq" podStartSLOduration=2.1038610110000002 podStartE2EDuration="4.489179339s" totalImagesPullingTime="2.385318328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:24.487328416 +0000 UTC m=+85.023283979" watchObservedRunningTime="2026-09-10 15:17:24.489179339 +0000 UTC m=+85.025134900"
Sep 10 15:17:25.423408 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:17:25.423380    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"prometheus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b1311c2217d4fd7cb932b0ce7682ead73b65f6c0211a54241e7fd807135c5cd3\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d"
Sep 10 15:17:26.243008 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:26.242968    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74b4cd8b4d-45hnw"
Sep 10 15:17:26.448100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:26.448046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:26.549193 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:26.549111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:26.606498 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:26.606446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:27.283253 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:27.283219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:27.380483 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:27.380420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:27.436235 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:27.436204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:27.571850 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:27.571819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:27.948062 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:27.947980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:28.013368 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:28.013319    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:28.149823 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:28.149794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:28.257322 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:28.257243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:28.761040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:28.761005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:28.828135 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:28.828101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:28.885152 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:28.885121    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:28.987811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:28.987778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:29.169102 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:29.169070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:29.423877 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:29.423798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-55865655fd-78bnb"
Sep 10 15:17:29.633899 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:29.633864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:29.780706 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:29.780625    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:29.839200 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:29.839168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:33.132133 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:33.132097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:33.229860 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:33.229832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:36.269983 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:36.269948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:36.332735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:36.332700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:39.737432 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:39.737396    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:39.795338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:39.795310    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:39.795432 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:39.795362    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:39.796421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:39.796398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:42.204925 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:42.204889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:42.264290 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:42.264258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:43.286010 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:43.285972    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dhxd7"
Sep 10 15:17:45.512768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:45.512723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerStarted","Data":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"}
Sep 10 15:17:45.548058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:45.547974    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.101663685 podStartE2EDuration="24.547955601s" totalImagesPullingTime="21.423081548s" totalInitContainerRuntime="23.210368ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-10 15:17:45.544043794 +0000 UTC m=+106.079999367" watchObservedRunningTime="2026-09-10 15:17:45.547955601 +0000 UTC m=+106.083911163"
Sep 10 15:17:45.984387 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:45.984349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:46.085397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:46.085357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:46.935008 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:46.934958    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:17:50.761587 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:50.761545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:50.823291 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:50.823261    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:51.283339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:51.283309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:51.384516 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:51.384478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:55.212402 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:55.212364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:55.303201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:55.303164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:56.167468 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:56.167414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:56.226314 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:56.226283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:17:59.801446 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:59.801410    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:17:59.805439 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:17:59.805409    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-559bd765d7-9fdj2"
Sep 10 15:18:00.174822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:00.174789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:00.235200 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:00.235166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:00.291506 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:00.291473    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:21.272014 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:21.271976    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:21.333899 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:21.333860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:21.934572 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:21.934538    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:21.955853 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:21.955828    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:22.689833 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:22.689803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:31.503637 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:31.503600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:31.553115 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:31.553083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:34.053272 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:34.053235    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:34.124188 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:34.124149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:34.224914 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:34.224880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:34.282331 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:34.282302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:34.800282 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:34.800240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:34.859086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:34.859052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:35.754136 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:35.754098    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:18:35.754577 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:35.754381    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="alertmanager" containerID="cri-o://c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1" gracePeriod=120
Sep 10 15:18:35.754577 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:35.754466    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-metric" containerID="cri-o://129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b" gracePeriod=120
Sep 10 15:18:35.754577 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:35.754488    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="prom-label-proxy" containerID="cri-o://b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4" gracePeriod=120
Sep 10 15:18:35.754577 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:35.754514    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy" containerID="cri-o://6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9" gracePeriod=120
Sep 10 15:18:35.754577 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:35.754529    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-web" containerID="cri-o://0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69" gracePeriod=120
Sep 10 15:18:35.754838 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:35.754542    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="config-reloader" containerID="cri-o://00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186" gracePeriod=120
Sep 10 15:18:36.735736 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735700    2562 generic.go:356] "Generic (PLEG): container finished" podID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerID="b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4" exitCode=0
Sep 10 15:18:36.735736 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735726    2562 generic.go:356] "Generic (PLEG): container finished" podID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerID="6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9" exitCode=0
Sep 10 15:18:36.735736 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735733    2562 generic.go:356] "Generic (PLEG): container finished" podID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerID="00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186" exitCode=0
Sep 10 15:18:36.735736 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735738    2562 generic.go:356] "Generic (PLEG): container finished" podID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerID="c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1" exitCode=0
Sep 10 15:18:36.736096 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735777    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"}
Sep 10 15:18:36.736096 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735858    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"}
Sep 10 15:18:36.736096 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"}
Sep 10 15:18:36.736096 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:36.735936    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"}
Sep 10 15:18:37.000206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.000151    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.045641 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045613    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-tls-assets\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.045894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045654    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-metrics-client-ca\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.045894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045671    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-web-config\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.045894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045705    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-web\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.045894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045732    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.045894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045793    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.045894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045829    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-main-db\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.045894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045872    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.046261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045902    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-out\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.046261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045954    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp5xc\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-kube-api-access-vp5xc\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.046261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.045994    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-trusted-ca-bundle\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.046261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.046035    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-cluster-tls-config\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.046261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.046065    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-volume\") pod \"ecb4b823-69ba-4acb-a114-750d4dcf318d\" (UID: \"ecb4b823-69ba-4acb-a114-750d4dcf318d\") "
Sep 10 15:18:37.046261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.046133    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:37.046582 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.046491    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-metrics-client-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.047817 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.047783    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:18:37.047928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.047869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:37.049922 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.049855    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:37.050012 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.049948    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-volume" (OuterVolumeSpecName: "config-volume") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:37.050261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.050219    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-out" (OuterVolumeSpecName: "config-out") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:18:37.051239 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.051197    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:18:37.051397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.051369    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-kube-api-access-vp5xc" (OuterVolumeSpecName: "kube-api-access-vp5xc") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "kube-api-access-vp5xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:18:37.051768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.051731    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:37.051954 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.051929    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:37.052043 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.052022    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:37.056335 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.056194    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:37.063093 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.063065    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-web-config" (OuterVolumeSpecName: "web-config") pod "ecb4b823-69ba-4acb-a114-750d4dcf318d" (UID: "ecb4b823-69ba-4acb-a114-750d4dcf318d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:37.147720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147666    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-web-config\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147714    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147726    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-main-tls\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147720 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147736    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147746    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-main-db\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147756    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147764    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-out\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147773    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp5xc\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-kube-api-access-vp5xc\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147781    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecb4b823-69ba-4acb-a114-750d4dcf318d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147789    2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-cluster-tls-config\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147799    2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecb4b823-69ba-4acb-a114-750d4dcf318d-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.147999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.147807    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecb4b823-69ba-4acb-a114-750d4dcf318d-tls-assets\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:37.742425 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.742388    2562 generic.go:356] "Generic (PLEG): container finished" podID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerID="129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b" exitCode=0
Sep 10 15:18:37.742425 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.742418    2562 generic.go:356] "Generic (PLEG): container finished" podID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerID="0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69" exitCode=0
Sep 10 15:18:37.742640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.742480    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.742640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.742480    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"}
Sep 10 15:18:37.742640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.742638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"}
Sep 10 15:18:37.742740 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.742669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecb4b823-69ba-4acb-a114-750d4dcf318d","Type":"ContainerDied","Data":"e1fde61f89ac9123e7c94c3ed120bbcdcf70ca0a0238f83706c4ae5c4a5de353"}
Sep 10 15:18:37.742740 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.742704    2562 scope.go:117] "RemoveContainer" containerID="b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"
Sep 10 15:18:37.751751 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.751728    2562 scope.go:117] "RemoveContainer" containerID="129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"
Sep 10 15:18:37.759043 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.759024    2562 scope.go:117] "RemoveContainer" containerID="6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"
Sep 10 15:18:37.765950 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.765930    2562 scope.go:117] "RemoveContainer" containerID="0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"
Sep 10 15:18:37.769111 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.769080    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:18:37.774320 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.774300    2562 scope.go:117] "RemoveContainer" containerID="00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"
Sep 10 15:18:37.775329 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.775308    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:18:37.781804 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.781789    2562 scope.go:117] "RemoveContainer" containerID="c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"
Sep 10 15:18:37.788866 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.788849    2562 scope.go:117] "RemoveContainer" containerID="811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396"
Sep 10 15:18:37.796276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.796259    2562 scope.go:117] "RemoveContainer" containerID="b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"
Sep 10 15:18:37.796627 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:37.796605    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4\": container with ID starting with b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4 not found: ID does not exist" containerID="b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"
Sep 10 15:18:37.796686 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.796634    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"} err="failed to get container status \"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4\": rpc error: code = NotFound desc = could not find container \"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4\": container with ID starting with b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4 not found: ID does not exist"
Sep 10 15:18:37.796686 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.796670    2562 scope.go:117] "RemoveContainer" containerID="129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"
Sep 10 15:18:37.796905 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:37.796884    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b\": container with ID starting with 129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b not found: ID does not exist" containerID="129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"
Sep 10 15:18:37.796962 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.796917    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"} err="failed to get container status \"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b\": rpc error: code = NotFound desc = could not find container \"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b\": container with ID starting with 129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b not found: ID does not exist"
Sep 10 15:18:37.796962 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.796942    2562 scope.go:117] "RemoveContainer" containerID="6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"
Sep 10 15:18:37.797179 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:37.797162    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9\": container with ID starting with 6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9 not found: ID does not exist" containerID="6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"
Sep 10 15:18:37.797219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.797183    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"} err="failed to get container status \"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9\": rpc error: code = NotFound desc = could not find container \"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9\": container with ID starting with 6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9 not found: ID does not exist"
Sep 10 15:18:37.797219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.797197    2562 scope.go:117] "RemoveContainer" containerID="0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"
Sep 10 15:18:37.797425 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:37.797410    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69\": container with ID starting with 0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69 not found: ID does not exist" containerID="0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"
Sep 10 15:18:37.797515 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.797430    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"} err="failed to get container status \"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69\": rpc error: code = NotFound desc = could not find container \"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69\": container with ID starting with 0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69 not found: ID does not exist"
Sep 10 15:18:37.797515 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.797463    2562 scope.go:117] "RemoveContainer" containerID="00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"
Sep 10 15:18:37.797729 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:37.797712    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186\": container with ID starting with 00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186 not found: ID does not exist" containerID="00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"
Sep 10 15:18:37.797811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.797735    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"} err="failed to get container status \"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186\": rpc error: code = NotFound desc = could not find container \"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186\": container with ID starting with 00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186 not found: ID does not exist"
Sep 10 15:18:37.797811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.797749    2562 scope.go:117] "RemoveContainer" containerID="c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"
Sep 10 15:18:37.797983 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:37.797967    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1\": container with ID starting with c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1 not found: ID does not exist" containerID="c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"
Sep 10 15:18:37.798022 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.797987    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"} err="failed to get container status \"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1\": rpc error: code = NotFound desc = could not find container \"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1\": container with ID starting with c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1 not found: ID does not exist"
Sep 10 15:18:37.798022 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798000    2562 scope.go:117] "RemoveContainer" containerID="811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396"
Sep 10 15:18:37.798239 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:37.798225    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396\": container with ID starting with 811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396 not found: ID does not exist" containerID="811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396"
Sep 10 15:18:37.798291 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798241    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396"} err="failed to get container status \"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396\": rpc error: code = NotFound desc = could not find container \"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396\": container with ID starting with 811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396 not found: ID does not exist"
Sep 10 15:18:37.798291 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798252    2562 scope.go:117] "RemoveContainer" containerID="b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"
Sep 10 15:18:37.798472 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798438    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4"} err="failed to get container status \"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4\": rpc error: code = NotFound desc = could not find container \"b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4\": container with ID starting with b7ab85dbd5cf7de36f38134bd4fc202ba07278f07a01ae2871c53071ca684ab4 not found: ID does not exist"
Sep 10 15:18:37.798524 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798481    2562 scope.go:117] "RemoveContainer" containerID="129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"
Sep 10 15:18:37.798709 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798690    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b"} err="failed to get container status \"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b\": rpc error: code = NotFound desc = could not find container \"129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b\": container with ID starting with 129b7cf9b1406ef2ebdfc0a811694bf7c7714e472edbe86846accd037533753b not found: ID does not exist"
Sep 10 15:18:37.798709 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798709    2562 scope.go:117] "RemoveContainer" containerID="6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"
Sep 10 15:18:37.798901 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798886    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9"} err="failed to get container status \"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9\": rpc error: code = NotFound desc = could not find container \"6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9\": container with ID starting with 6e9ad21e354a463d88ffc5f829fe29a20158f076f731d1c92b43fb29baaa9cb9 not found: ID does not exist"
Sep 10 15:18:37.798947 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.798900    2562 scope.go:117] "RemoveContainer" containerID="0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"
Sep 10 15:18:37.799131 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.799112    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69"} err="failed to get container status \"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69\": rpc error: code = NotFound desc = could not find container \"0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69\": container with ID starting with 0522100aee96b46b9f9ae675fd585bb74428b52d571c3d1a2a2e47c8cf1b3f69 not found: ID does not exist"
Sep 10 15:18:37.799169 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.799132    2562 scope.go:117] "RemoveContainer" containerID="00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"
Sep 10 15:18:37.799327 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.799307    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186"} err="failed to get container status \"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186\": rpc error: code = NotFound desc = could not find container \"00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186\": container with ID starting with 00f608945b9e9782a02f13700157b16f91624dabd3369709786caafcbd0b0186 not found: ID does not exist"
Sep 10 15:18:37.799327 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.799319    2562 scope.go:117] "RemoveContainer" containerID="c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"
Sep 10 15:18:37.799558 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.799536    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1"} err="failed to get container status \"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1\": rpc error: code = NotFound desc = could not find container \"c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1\": container with ID starting with c730430d1d6631c1992b60e4a68ffc6f6a617924cfa6d511ae8683315539bca1 not found: ID does not exist"
Sep 10 15:18:37.799607 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.799558    2562 scope.go:117] "RemoveContainer" containerID="811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396"
Sep 10 15:18:37.799799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.799778    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396"} err="failed to get container status \"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396\": rpc error: code = NotFound desc = could not find container \"811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396\": container with ID starting with 811d55bba77b44256f516f5b640ba93bd4fe19e5a79d52376eba214adac5b396 not found: ID does not exist"
Sep 10 15:18:37.804746 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.804724    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:18:37.805034 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805021    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-metric"
Sep 10 15:18:37.805076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805035    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-metric"
Sep 10 15:18:37.805076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805047    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy"
Sep 10 15:18:37.805076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805052    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy"
Sep 10 15:18:37.805076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805059    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-web"
Sep 10 15:18:37.805076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805065    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-web"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805078    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="alertmanager"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805084    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="alertmanager"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805094    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="init-config-reloader"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805099    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="init-config-reloader"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805106    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="prom-label-proxy"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805111    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="prom-label-proxy"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805118    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="config-reloader"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805123    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="config-reloader"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805180    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805187    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="prom-label-proxy"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805195    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="alertmanager"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805200    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-web"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805207    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="config-reloader"
Sep 10 15:18:37.805220 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.805215    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" containerName="kube-rbac-proxy-metric"
Sep 10 15:18:37.810135 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.810116    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.815555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.815531    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 15:18:37.815839 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.815816    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 15:18:37.815968 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.815857    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 15:18:37.815968 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.815907    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 15:18:37.815968 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.815907    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 15:18:37.815968 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.815929    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9ntxh\""
Sep 10 15:18:37.815968 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.815926    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 15:18:37.816288 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.816235    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 15:18:37.816288 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.816281    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 15:18:37.823400 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.823372    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 15:18:37.838225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.838188    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:18:37.852000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.851960    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.851999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/488bb619-59df-4365-9ff1-4736a58eb8f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852082    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-web-config\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/488bb619-59df-4365-9ff1-4736a58eb8f3-config-out\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/488bb619-59df-4365-9ff1-4736a58eb8f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/488bb619-59df-4365-9ff1-4736a58eb8f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852285    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/488bb619-59df-4365-9ff1-4736a58eb8f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852335    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx9s8\" (UniqueName: \"kubernetes.io/projected/488bb619-59df-4365-9ff1-4736a58eb8f3-kube-api-access-sx9s8\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.852397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.852359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.952866 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.952828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.952875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx9s8\" (UniqueName: \"kubernetes.io/projected/488bb619-59df-4365-9ff1-4736a58eb8f3-kube-api-access-sx9s8\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.952900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.952939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.952963    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/488bb619-59df-4365-9ff1-4736a58eb8f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953052    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-web-config\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/488bb619-59df-4365-9ff1-4736a58eb8f3-config-out\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/488bb619-59df-4365-9ff1-4736a58eb8f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/488bb619-59df-4365-9ff1-4736a58eb8f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/488bb619-59df-4365-9ff1-4736a58eb8f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.953739 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.953670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/488bb619-59df-4365-9ff1-4736a58eb8f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.954820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.954523    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/488bb619-59df-4365-9ff1-4736a58eb8f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.955894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.955862    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/488bb619-59df-4365-9ff1-4736a58eb8f3-config-out\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.956140 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.956119    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.956229 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.956125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.956229 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.956150    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecb4b823-69ba-4acb-a114-750d4dcf318d" path="/var/lib/kubelet/pods/ecb4b823-69ba-4acb-a114-750d4dcf318d/volumes"
Sep 10 15:18:37.956229 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.956174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.956699 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.956647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/488bb619-59df-4365-9ff1-4736a58eb8f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.956801 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.956717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/488bb619-59df-4365-9ff1-4736a58eb8f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.957337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.957309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.957410 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.957317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.958243 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.958219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-web-config\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.958497 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.958442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/488bb619-59df-4365-9ff1-4736a58eb8f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:37.963531 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:37.963505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx9s8\" (UniqueName: \"kubernetes.io/projected/488bb619-59df-4365-9ff1-4736a58eb8f3-kube-api-access-sx9s8\") pod \"alertmanager-main-0\" (UID: \"488bb619-59df-4365-9ff1-4736a58eb8f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:38.119405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:38.119370    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 15:18:38.421964 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:38.421937    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 15:18:38.423780 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:18:38.423744    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488bb619_59df_4365_9ff1_4736a58eb8f3.slice/crio-99165b9f07be53fffa0a0fb6528b626ac02c9e69d00fa9bc77e335c2e0371c0f WatchSource:0}: Error finding container 99165b9f07be53fffa0a0fb6528b626ac02c9e69d00fa9bc77e335c2e0371c0f: Status 404 returned error can't find the container with id 99165b9f07be53fffa0a0fb6528b626ac02c9e69d00fa9bc77e335c2e0371c0f
Sep 10 15:18:38.748170 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:38.748062    2562 generic.go:356] "Generic (PLEG): container finished" podID="488bb619-59df-4365-9ff1-4736a58eb8f3" containerID="6ca40d086a19528b5b86758b75810ce67c1ebe90c0d6a5568e599451ad00419b" exitCode=0
Sep 10 15:18:38.748170 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:38.748159    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerDied","Data":"6ca40d086a19528b5b86758b75810ce67c1ebe90c0d6a5568e599451ad00419b"}
Sep 10 15:18:38.748373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:38.748224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerStarted","Data":"99165b9f07be53fffa0a0fb6528b626ac02c9e69d00fa9bc77e335c2e0371c0f"}
Sep 10 15:18:39.756100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:39.756063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerStarted","Data":"ad6829c1f305bc86f9fca69d68f52ad56bc31df82db157eb3bc33cb427b33f20"}
Sep 10 15:18:39.756491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:39.756173    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerStarted","Data":"763166b5480615713fc0bec37212a43944c6e4f4ee67b5f9433ee142479a35ee"}
Sep 10 15:18:39.756491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:39.756206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerStarted","Data":"e4bc793dd371b8e932725fa98f7f9a57c7c19e5c1e2436896991f65af4868412"}
Sep 10 15:18:39.756491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:39.756231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerStarted","Data":"33cd470e59195370c3aa3216461a88619e631870dae788bf0f3ee3548f4982e0"}
Sep 10 15:18:39.756491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:39.756267    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerStarted","Data":"bf15183badfd1346c3319aba5bbf1d860795ae822957fe20be19994fdf59b0aa"}
Sep 10 15:18:39.756491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:39.756292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"488bb619-59df-4365-9ff1-4736a58eb8f3","Type":"ContainerStarted","Data":"c7d9a1236e85a4ebef19c9097a0f74493355c8f02999f951cab88e77dfc40bf3"}
Sep 10 15:18:39.788787 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:39.788665    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.774214368 podStartE2EDuration="2.78864125s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.426882ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:18:39.786488049 +0000 UTC m=+160.322443635" watchObservedRunningTime="2026-09-10 15:18:39.78864125 +0000 UTC m=+160.324596812"
Sep 10 15:18:40.024805 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.024692    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:18:40.025143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.025063    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1" gracePeriod=600
Sep 10 15:18:40.025143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.025084    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="prometheus" containerID="cri-o://c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd" gracePeriod=600
Sep 10 15:18:40.025143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.025106    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="thanos-sidecar" containerID="cri-o://6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691" gracePeriod=600
Sep 10 15:18:40.025143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.025114    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-web" containerID="cri-o://2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4" gracePeriod=600
Sep 10 15:18:40.025388 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.025045    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="config-reloader" containerID="cri-o://750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0" gracePeriod=600
Sep 10 15:18:40.025388 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.025106    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy" containerID="cri-o://0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9" gracePeriod=600
Sep 10 15:18:40.270016 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.269989    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.373609 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373577    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-thanos-prometheus-http-client-file\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.373849 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373628    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.373849 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373648    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-trusted-ca-bundle\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.373849 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373668    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4s2g\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-kube-api-access-r4s2g\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.373849 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373686    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-kubelet-serving-ca-bundle\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-grpc-tls\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373895    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-0\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373934    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-1\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373963    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-metrics-client-ca\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.373989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-serving-certs-ca-bundle\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374026    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-metrics-client-certs\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374030    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374059    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-web-config\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374082    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374099    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374118    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-tls-assets\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374144    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config-out\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374183    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-db\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374227    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-2\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374249    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-kube-rbac-proxy\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374272    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374308    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-tls\") pod \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\" (UID: \"cd82cc40-4433-4a70-96d5-99c7d57ba35d\") "
Sep 10 15:18:40.374542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374371    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:40.374976 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374645    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.374976 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374668    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.374976 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.374684    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.375584 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.375552    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:40.376276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376178    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.376276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376246    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.376474 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376431    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-kube-api-access-r4s2g" (OuterVolumeSpecName: "kube-api-access-r4s2g") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "kube-api-access-r4s2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:18:40.376556 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376515    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.376645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376620    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:40.376929 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376901    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.377011 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376952    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config-out" (OuterVolumeSpecName: "config-out") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:18:40.377011 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.376958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:40.377368 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.377341    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:18:40.377658 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.377632    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:18:40.377886 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.377863    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config" (OuterVolumeSpecName: "config") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.378085 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.378069    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.378493 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.378444    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.379148 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.379131    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.379198 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.379143    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:18:40.389114 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.389095    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-web-config" (OuterVolumeSpecName: "web-config") pod "cd82cc40-4433-4a70-96d5-99c7d57ba35d" (UID: "cd82cc40-4433-4a70-96d5-99c7d57ba35d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:18:40.475351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475259    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-grpc-tls\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475300    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475322    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-metrics-client-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475337    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475352    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-metrics-client-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475363    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-web-config\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475376    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475387    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-tls-assets\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475397    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-config-out\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475408    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-db\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475421    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cd82cc40-4433-4a70-96d5-99c7d57ba35d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475433    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-kube-rbac-proxy\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475445    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475476    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475488    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475502    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd82cc40-4433-4a70-96d5-99c7d57ba35d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.475734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.475516    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4s2g\" (UniqueName: \"kubernetes.io/projected/cd82cc40-4433-4a70-96d5-99c7d57ba35d-kube-api-access-r4s2g\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:18:40.763165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763078    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd" exitCode=0
Sep 10 15:18:40.763165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763103    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1" exitCode=0
Sep 10 15:18:40.763165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763114    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9" exitCode=0
Sep 10 15:18:40.763165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763120    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4" exitCode=0
Sep 10 15:18:40.763165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763125    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691" exitCode=0
Sep 10 15:18:40.763165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763130    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0" exitCode=0
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763171    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"}
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763306    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"}
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763334    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"}
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763362    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"}
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763370    2562 scope.go:117] "RemoveContainer" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763387    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"}
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"}
Sep 10 15:18:40.763862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.763435    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cd82cc40-4433-4a70-96d5-99c7d57ba35d","Type":"ContainerDied","Data":"514942b8908ee2428e29c8cf3ac34572384323cc30da29867317bc0769ab304b"}
Sep 10 15:18:40.773075 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.773057    2562 scope.go:117] "RemoveContainer" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.780413 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.780390    2562 scope.go:117] "RemoveContainer" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.792837 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.792136    2562 scope.go:117] "RemoveContainer" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.792837 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.792219    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:18:40.798107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.798078    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:18:40.802876 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.802855    2562 scope.go:117] "RemoveContainer" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.809615 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.809595    2562 scope.go:117] "RemoveContainer" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.816257 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.816241    2562 scope.go:117] "RemoveContainer" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.824304 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824283    2562 scope.go:117] "RemoveContainer" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.824398 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824335    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:18:40.824670 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:40.824650    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": container with ID starting with c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd not found: ID does not exist" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.824729 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824681    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"} err="failed to get container status \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": rpc error: code = NotFound desc = could not find container \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": container with ID starting with c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd not found: ID does not exist"
Sep 10 15:18:40.824729 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824708    2562 scope.go:117] "RemoveContainer" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824732    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="init-config-reloader"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824746    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="init-config-reloader"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824755    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="thanos-sidecar"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824761    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="thanos-sidecar"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824771    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-thanos"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824777    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-thanos"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824793    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="config-reloader"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824799    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="config-reloader"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824806    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="prometheus"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824811    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="prometheus"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824820    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-web"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824826    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-web"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824832    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy"
Sep 10 15:18:40.824829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824837    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824889    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="config-reloader"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824898    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824905    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="thanos-sidecar"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824911    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-web"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824915    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="prometheus"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824921    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" containerName="kube-rbac-proxy-thanos"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:40.824974    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": container with ID starting with 0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1 not found: ID does not exist" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.824991    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"} err="failed to get container status \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": rpc error: code = NotFound desc = could not find container \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": container with ID starting with 0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1 not found: ID does not exist"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.825004    2562 scope.go:117] "RemoveContainer" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:40.825215    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": container with ID starting with 0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9 not found: ID does not exist" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.825230    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"} err="failed to get container status \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": rpc error: code = NotFound desc = could not find container \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": container with ID starting with 0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9 not found: ID does not exist"
Sep 10 15:18:40.825310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.825252    2562 scope.go:117] "RemoveContainer" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.825743 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:40.825529    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": container with ID starting with 2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4 not found: ID does not exist" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.825743 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.825560    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"} err="failed to get container status \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": rpc error: code = NotFound desc = could not find container \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": container with ID starting with 2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4 not found: ID does not exist"
Sep 10 15:18:40.825743 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.825573    2562 scope.go:117] "RemoveContainer" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.825836 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:40.825783    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": container with ID starting with 6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691 not found: ID does not exist" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.825836 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.825798    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"} err="failed to get container status \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": rpc error: code = NotFound desc = could not find container \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": container with ID starting with 6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691 not found: ID does not exist"
Sep 10 15:18:40.825836 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.825810    2562 scope.go:117] "RemoveContainer" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.826041 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:40.826026    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": container with ID starting with 750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0 not found: ID does not exist" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.826076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826044    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"} err="failed to get container status \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": rpc error: code = NotFound desc = could not find container \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": container with ID starting with 750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0 not found: ID does not exist"
Sep 10 15:18:40.826076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826055    2562 scope.go:117] "RemoveContainer" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.826286 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:18:40.826267    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": container with ID starting with ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88 not found: ID does not exist" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.826337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826289    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"} err="failed to get container status \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": rpc error: code = NotFound desc = could not find container \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": container with ID starting with ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88 not found: ID does not exist"
Sep 10 15:18:40.826337 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826302    2562 scope.go:117] "RemoveContainer" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.826576 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826559    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"} err="failed to get container status \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": rpc error: code = NotFound desc = could not find container \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": container with ID starting with c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd not found: ID does not exist"
Sep 10 15:18:40.826576 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826575    2562 scope.go:117] "RemoveContainer" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.826802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826781    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"} err="failed to get container status \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": rpc error: code = NotFound desc = could not find container \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": container with ID starting with 0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1 not found: ID does not exist"
Sep 10 15:18:40.826849 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826802    2562 scope.go:117] "RemoveContainer" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.827008 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.826992    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"} err="failed to get container status \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": rpc error: code = NotFound desc = could not find container \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": container with ID starting with 0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9 not found: ID does not exist"
Sep 10 15:18:40.827046 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827008    2562 scope.go:117] "RemoveContainer" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.827231 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827215    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"} err="failed to get container status \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": rpc error: code = NotFound desc = could not find container \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": container with ID starting with 2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4 not found: ID does not exist"
Sep 10 15:18:40.827231 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827231    2562 scope.go:117] "RemoveContainer" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.827544 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827508    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"} err="failed to get container status \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": rpc error: code = NotFound desc = could not find container \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": container with ID starting with 6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691 not found: ID does not exist"
Sep 10 15:18:40.827544 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827545    2562 scope.go:117] "RemoveContainer" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.827804 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827786    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"} err="failed to get container status \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": rpc error: code = NotFound desc = could not find container \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": container with ID starting with 750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0 not found: ID does not exist"
Sep 10 15:18:40.827848 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827806    2562 scope.go:117] "RemoveContainer" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.828015 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.827994    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"} err="failed to get container status \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": rpc error: code = NotFound desc = could not find container \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": container with ID starting with ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88 not found: ID does not exist"
Sep 10 15:18:40.828082 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828015    2562 scope.go:117] "RemoveContainer" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.828258 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828241    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"} err="failed to get container status \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": rpc error: code = NotFound desc = could not find container \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": container with ID starting with c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd not found: ID does not exist"
Sep 10 15:18:40.828320 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828261    2562 scope.go:117] "RemoveContainer" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.828491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828473    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"} err="failed to get container status \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": rpc error: code = NotFound desc = could not find container \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": container with ID starting with 0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1 not found: ID does not exist"
Sep 10 15:18:40.828535 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828491    2562 scope.go:117] "RemoveContainer" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.828693 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828675    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"} err="failed to get container status \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": rpc error: code = NotFound desc = could not find container \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": container with ID starting with 0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9 not found: ID does not exist"
Sep 10 15:18:40.828741 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828694    2562 scope.go:117] "RemoveContainer" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.828902 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828885    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"} err="failed to get container status \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": rpc error: code = NotFound desc = could not find container \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": container with ID starting with 2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4 not found: ID does not exist"
Sep 10 15:18:40.828966 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.828904    2562 scope.go:117] "RemoveContainer" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.829133 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829117    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"} err="failed to get container status \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": rpc error: code = NotFound desc = could not find container \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": container with ID starting with 6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691 not found: ID does not exist"
Sep 10 15:18:40.829177 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829133    2562 scope.go:117] "RemoveContainer" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.829320 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829298    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"} err="failed to get container status \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": rpc error: code = NotFound desc = could not find container \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": container with ID starting with 750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0 not found: ID does not exist"
Sep 10 15:18:40.829363 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829322    2562 scope.go:117] "RemoveContainer" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.829512 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829496    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"} err="failed to get container status \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": rpc error: code = NotFound desc = could not find container \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": container with ID starting with ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88 not found: ID does not exist"
Sep 10 15:18:40.829512 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829511    2562 scope.go:117] "RemoveContainer" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.829697 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829679    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"} err="failed to get container status \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": rpc error: code = NotFound desc = could not find container \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": container with ID starting with c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd not found: ID does not exist"
Sep 10 15:18:40.829756 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829698    2562 scope.go:117] "RemoveContainer" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.829899 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829881    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"} err="failed to get container status \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": rpc error: code = NotFound desc = could not find container \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": container with ID starting with 0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1 not found: ID does not exist"
Sep 10 15:18:40.829948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.829901    2562 scope.go:117] "RemoveContainer" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.830131 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830109    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"} err="failed to get container status \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": rpc error: code = NotFound desc = could not find container \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": container with ID starting with 0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9 not found: ID does not exist"
Sep 10 15:18:40.830173 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830132    2562 scope.go:117] "RemoveContainer" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.830362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830344    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"} err="failed to get container status \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": rpc error: code = NotFound desc = could not find container \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": container with ID starting with 2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4 not found: ID does not exist"
Sep 10 15:18:40.830400 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830363    2562 scope.go:117] "RemoveContainer" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.830648 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830632    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.830648 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830641    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"} err="failed to get container status \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": rpc error: code = NotFound desc = could not find container \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": container with ID starting with 6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691 not found: ID does not exist"
Sep 10 15:18:40.830783 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830655    2562 scope.go:117] "RemoveContainer" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.831029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830845    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"} err="failed to get container status \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": rpc error: code = NotFound desc = could not find container \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": container with ID starting with 750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0 not found: ID does not exist"
Sep 10 15:18:40.831029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.830872    2562 scope.go:117] "RemoveContainer" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.831168 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831117    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"} err="failed to get container status \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": rpc error: code = NotFound desc = could not find container \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": container with ID starting with ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88 not found: ID does not exist"
Sep 10 15:18:40.831168 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831138    2562 scope.go:117] "RemoveContainer" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.831439 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831414    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"} err="failed to get container status \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": rpc error: code = NotFound desc = could not find container \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": container with ID starting with c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd not found: ID does not exist"
Sep 10 15:18:40.831439 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831439    2562 scope.go:117] "RemoveContainer" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.831711 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831692    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"} err="failed to get container status \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": rpc error: code = NotFound desc = could not find container \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": container with ID starting with 0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1 not found: ID does not exist"
Sep 10 15:18:40.831771 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831712    2562 scope.go:117] "RemoveContainer" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.831963 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831937    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"} err="failed to get container status \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": rpc error: code = NotFound desc = could not find container \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": container with ID starting with 0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9 not found: ID does not exist"
Sep 10 15:18:40.831963 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.831962    2562 scope.go:117] "RemoveContainer" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.832215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832198    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"} err="failed to get container status \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": rpc error: code = NotFound desc = could not find container \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": container with ID starting with 2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4 not found: ID does not exist"
Sep 10 15:18:40.832257 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832216    2562 scope.go:117] "RemoveContainer" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.832475 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832441    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"} err="failed to get container status \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": rpc error: code = NotFound desc = could not find container \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": container with ID starting with 6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691 not found: ID does not exist"
Sep 10 15:18:40.832540 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832477    2562 scope.go:117] "RemoveContainer" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.832712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832695    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"} err="failed to get container status \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": rpc error: code = NotFound desc = could not find container \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": container with ID starting with 750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0 not found: ID does not exist"
Sep 10 15:18:40.832761 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832713    2562 scope.go:117] "RemoveContainer" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.832933 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832915    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"} err="failed to get container status \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": rpc error: code = NotFound desc = could not find container \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": container with ID starting with ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88 not found: ID does not exist"
Sep 10 15:18:40.833002 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.832935    2562 scope.go:117] "RemoveContainer" containerID="c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"
Sep 10 15:18:40.833165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833146    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd"} err="failed to get container status \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": rpc error: code = NotFound desc = could not find container \"c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd\": container with ID starting with c930be3e1d67de15edeb811e58506ae01bab86b9f87259c12b557d261463ddbd not found: ID does not exist"
Sep 10 15:18:40.833238 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833166    2562 scope.go:117] "RemoveContainer" containerID="0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"
Sep 10 15:18:40.833496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833368    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1"} err="failed to get container status \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": rpc error: code = NotFound desc = could not find container \"0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1\": container with ID starting with 0b4b5e656cec1aca1d4a5a931d914313df4c972027bdb9e49fe9baf8891886a1 not found: ID does not exist"
Sep 10 15:18:40.833496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833393    2562 scope.go:117] "RemoveContainer" containerID="0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"
Sep 10 15:18:40.833496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833440    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5wmhw\""
Sep 10 15:18:40.833689 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833440    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 15:18:40.833689 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833440    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 15:18:40.833806 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833789    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 15:18:40.833860 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833782    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9"} err="failed to get container status \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": rpc error: code = NotFound desc = could not find container \"0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9\": container with ID starting with 0b2ff66000dfd9d2bf2f932f721ccbf558103111632d141c4b636634ffe9c3e9 not found: ID does not exist"
Sep 10 15:18:40.833860 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833840    2562 scope.go:117] "RemoveContainer" containerID="2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"
Sep 10 15:18:40.833970 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833888    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cpi4rrse0opub\""
Sep 10 15:18:40.833970 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833901    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 15:18:40.833970 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.833936    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 15:18:40.834213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834184    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4"} err="failed to get container status \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": rpc error: code = NotFound desc = could not find container \"2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4\": container with ID starting with 2166ab1307f17da297e678ef78c5f089900df0a65889ecff1954c72c7441c8d4 not found: ID does not exist"
Sep 10 15:18:40.834213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834213    2562 scope.go:117] "RemoveContainer" containerID="6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"
Sep 10 15:18:40.834378 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834229    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 15:18:40.834378 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834243    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 15:18:40.834378 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834195    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 15:18:40.834378 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834356    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 15:18:40.834601 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834488    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691"} err="failed to get container status \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": rpc error: code = NotFound desc = could not find container \"6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691\": container with ID starting with 6b69a73bf5ae7c883efa2f6d5dc1fbbe5b741713fa39dbe06264d08c53a9e691 not found: ID does not exist"
Sep 10 15:18:40.834601 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834510    2562 scope.go:117] "RemoveContainer" containerID="750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"
Sep 10 15:18:40.834791 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834763    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0"} err="failed to get container status \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": rpc error: code = NotFound desc = could not find container \"750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0\": container with ID starting with 750607910a6e01b44f5a5f1506f6d03b170b88768362d9cb1c60437be2ec72a0 not found: ID does not exist"
Sep 10 15:18:40.834844 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834794    2562 scope.go:117] "RemoveContainer" containerID="ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"
Sep 10 15:18:40.834948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.834931    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 15:18:40.835024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.835007    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 15:18:40.835187 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.835158    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88"} err="failed to get container status \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": rpc error: code = NotFound desc = could not find container \"ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88\": container with ID starting with ddbbb3554214820c5105a80b9bf3860fcddb59eb47d9106d0772ec07de4b5f88 not found: ID does not exist"
Sep 10 15:18:40.835418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.835401    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 15:18:40.836849 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.836832    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 15:18:40.838851 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.838832    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 15:18:40.847369 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.847349    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:18:40.879101 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v8gt\" (UniqueName: \"kubernetes.io/projected/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-kube-api-access-9v8gt\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879204    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879566 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879377    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879566 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879566 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879566 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879762 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879729    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879762 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879889 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879889 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879912    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-config\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879936    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.879974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.879958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.880068 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.880038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.880102 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.880085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.880136 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.880110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.980942 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.980906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-config\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.980942 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.980944    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.980962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.980990    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v8gt\" (UniqueName: \"kubernetes.io/projected/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-kube-api-access-9v8gt\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981324    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981598    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981958 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.981958 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.981777    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.982058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.982045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.982171 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.982146    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.983156 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.983127    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.983882 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.983852    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.984134 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.984108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-config\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.984225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.984109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.984225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.984173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.984338 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.984218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.984826 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.984519    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.985087 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.985062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.985164 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.985093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.985411 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.985387    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.986069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.986046    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.986360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.986335    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.986499 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.986481    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.986562 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.986537    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.986652 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.986634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.986972 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.986957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:40.989302 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:40.989280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v8gt\" (UniqueName: \"kubernetes.io/projected/43c3ad68-8719-4295-9bbc-b3fd15cd9f83-kube-api-access-9v8gt\") pod \"prometheus-k8s-0\" (UID: \"43c3ad68-8719-4295-9bbc-b3fd15cd9f83\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:41.141160 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:41.141114    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:41.424704 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:41.424671    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:18:41.427836 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:18:41.427801    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c3ad68_8719_4295_9bbc_b3fd15cd9f83.slice/crio-28fb9368af37cbf01bc3499a0c8a781d42a49eb4d89101c47ee13fa9bbfd5a7c WatchSource:0}: Error finding container 28fb9368af37cbf01bc3499a0c8a781d42a49eb4d89101c47ee13fa9bbfd5a7c: Status 404 returned error can't find the container with id 28fb9368af37cbf01bc3499a0c8a781d42a49eb4d89101c47ee13fa9bbfd5a7c
Sep 10 15:18:41.769542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:41.769410    2562 generic.go:356] "Generic (PLEG): container finished" podID="43c3ad68-8719-4295-9bbc-b3fd15cd9f83" containerID="d9afd081d6bfa5aee1d6c5c4302e12d6f785cb90924f888d1b1c7d56efc7a29c" exitCode=0
Sep 10 15:18:41.769542 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:41.769501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerDied","Data":"d9afd081d6bfa5aee1d6c5c4302e12d6f785cb90924f888d1b1c7d56efc7a29c"}
Sep 10 15:18:41.769957 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:41.769591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerStarted","Data":"28fb9368af37cbf01bc3499a0c8a781d42a49eb4d89101c47ee13fa9bbfd5a7c"}
Sep 10 15:18:41.959563 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:41.957792    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd82cc40-4433-4a70-96d5-99c7d57ba35d" path="/var/lib/kubelet/pods/cd82cc40-4433-4a70-96d5-99c7d57ba35d/volumes"
Sep 10 15:18:42.778401 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:42.778364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerStarted","Data":"66232df1dbc16ada6eb68a25407434a762e4f03ac5d2efd8c6fe4d8899fa04aa"}
Sep 10 15:18:42.778784 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:42.778485    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerStarted","Data":"8a873ba417f86194808ae423e15fa81bfd7ac7a30facf83b0d0a2ef93d48b58c"}
Sep 10 15:18:42.778784 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:42.778521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerStarted","Data":"bc6366517d1999ab2d6d99fb1f8a9ea46c30f7447866517bd5e018b098142d2d"}
Sep 10 15:18:42.778784 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:42.778546    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerStarted","Data":"e4ca2a992eeaa9a60ae96aeccb5446682e09287be89e43083c10ef4d1dd5b4a9"}
Sep 10 15:18:42.778784 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:42.778568    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerStarted","Data":"1ea2441b2ee1237cb3ef313b08c5a1524f7591a424c4b7c83620145592585cbb"}
Sep 10 15:18:42.778784 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:42.778591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c3ad68-8719-4295-9bbc-b3fd15cd9f83","Type":"ContainerStarted","Data":"77e04a89485097dc107fef8d8b6f67d09f9c213f207d5638ebf957705ecce237"}
Sep 10 15:18:42.809388 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:42.809330    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.790998735 podStartE2EDuration="2.809312373s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.313638ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:18:42.807325782 +0000 UTC m=+163.343281365" watchObservedRunningTime="2026-09-10 15:18:42.809312373 +0000 UTC m=+163.345267935"
Sep 10 15:18:43.499550 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:43.499516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:43.599295 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:43.599264    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:45.390334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:45.390280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:45.442519 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:45.442483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:45.500827 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:45.500799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:46.141293 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:46.141243    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:18:47.796613 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:47.796575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:51.275412 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:51.275384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:51.376868 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:51.376838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:18:51.523163 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:18:51.523126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:28.908841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:28.908804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:28.970366 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:28.970336    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:30.805578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:30.805541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:30.906027 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:30.905978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:41.141416 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:41.141375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:19:41.156841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:41.156812    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:19:42.037153 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:42.037126    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:19:52.747934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:52.747900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:53.108171 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:53.108127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:56.376088 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:56.376052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:56.476942 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:56.476907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:57.919870 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:57.919837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:19:58.035390 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:19:58.035351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:00.260550 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:00.260447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:00.682007 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:00.681972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:03.195082 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:03.195042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:03.295934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:03.295895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:05.050186 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:05.050151    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:05.959730 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:05.959700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:06.092701 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:06.092663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:07.490055 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:07.490020    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:08.556776 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:08.556739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:08.659737 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:08.659702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:14.891513 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:14.891476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:14.997536 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:14.997499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:17.359737 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:17.359700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:31.060652 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:31.060569    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:31.166632 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:31.166603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:31.694755 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:31.694715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:31.991435 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:31.991342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:32.083633 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.083600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:32.117775 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.117744    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"]
Sep 10 15:20:32.121060 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.121043    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.124192 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.124172    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 15:20:32.124274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.124255    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wbskq\""
Sep 10 15:20:32.124490 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.124474    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 15:20:32.124555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.124541    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 15:20:32.125165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.125146    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 15:20:32.141104 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.141069    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"]
Sep 10 15:20:32.255201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.255111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:32.263335 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.263309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c07bb165-1fe4-4340-97b4-956b0e1e6d25-manifests\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.263419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.263341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsk9l\" (UniqueName: \"kubernetes.io/projected/c07bb165-1fe4-4340-97b4-956b0e1e6d25-kube-api-access-hsk9l\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.263419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.263364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c07bb165-1fe4-4340-97b4-956b0e1e6d25-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.263555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.263535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c07bb165-1fe4-4340-97b4-956b0e1e6d25-tmp\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.263591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.263567    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07bb165-1fe4-4340-97b4-956b0e1e6d25-webhook-cert\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.356181 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.356145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:32.364401 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.364373    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c07bb165-1fe4-4340-97b4-956b0e1e6d25-tmp\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.364532 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.364406    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07bb165-1fe4-4340-97b4-956b0e1e6d25-webhook-cert\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.364596 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.364565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c07bb165-1fe4-4340-97b4-956b0e1e6d25-manifests\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.364596 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.364590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsk9l\" (UniqueName: \"kubernetes.io/projected/c07bb165-1fe4-4340-97b4-956b0e1e6d25-kube-api-access-hsk9l\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.364694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.364613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c07bb165-1fe4-4340-97b4-956b0e1e6d25-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.364954 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.364930    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c07bb165-1fe4-4340-97b4-956b0e1e6d25-manifests\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.371565 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.367188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c07bb165-1fe4-4340-97b4-956b0e1e6d25-tmp\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.371565 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.367864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c07bb165-1fe4-4340-97b4-956b0e1e6d25-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.372270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.372238    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07bb165-1fe4-4340-97b4-956b0e1e6d25-webhook-cert\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.374050 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.374014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsk9l\" (UniqueName: \"kubernetes.io/projected/c07bb165-1fe4-4340-97b4-956b0e1e6d25-kube-api-access-hsk9l\") pod \"opendatahub-operator-controller-manager-76f847f9cd-lz6z2\" (UID: \"c07bb165-1fe4-4340-97b4-956b0e1e6d25\") " pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.431931 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.431889    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:32.455081 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.455052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:32.552473 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.552371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:32.653129 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.653097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:32.726930 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:32.726887    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"]
Sep 10 15:20:32.729517 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:20:32.729484    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07bb165_1fe4_4340_97b4_956b0e1e6d25.slice/crio-76f2b3ca76a0d9fbbd1847c31fd5b3a95d8b0fb9862dc9fda37c23fb8af0b5c7 WatchSource:0}: Error finding container 76f2b3ca76a0d9fbbd1847c31fd5b3a95d8b0fb9862dc9fda37c23fb8af0b5c7: Status 404 returned error can't find the container with id 76f2b3ca76a0d9fbbd1847c31fd5b3a95d8b0fb9862dc9fda37c23fb8af0b5c7
Sep 10 15:20:33.228240 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:33.228207    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2" event={"ID":"c07bb165-1fe4-4340-97b4-956b0e1e6d25","Type":"ContainerStarted","Data":"76f2b3ca76a0d9fbbd1847c31fd5b3a95d8b0fb9862dc9fda37c23fb8af0b5c7"}
Sep 10 15:20:36.244629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:36.244593    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2" event={"ID":"c07bb165-1fe4-4340-97b4-956b0e1e6d25","Type":"ContainerStarted","Data":"dcd12ef50e6791355c2b8dd029cf8ce805f944b0bf0db8958425d5b6b9916e10"}
Sep 10 15:20:37.251076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:37.251040    2562 generic.go:356] "Generic (PLEG): container finished" podID="c07bb165-1fe4-4340-97b4-956b0e1e6d25" containerID="dcd12ef50e6791355c2b8dd029cf8ce805f944b0bf0db8958425d5b6b9916e10" exitCode=0
Sep 10 15:20:37.251486 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:37.251081    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2" event={"ID":"c07bb165-1fe4-4340-97b4-956b0e1e6d25","Type":"ContainerDied","Data":"dcd12ef50e6791355c2b8dd029cf8ce805f944b0bf0db8958425d5b6b9916e10"}
Sep 10 15:20:38.269858 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:38.269810    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2" event={"ID":"c07bb165-1fe4-4340-97b4-956b0e1e6d25","Type":"ContainerStarted","Data":"56015f074554cad43b1f2ea5883a8a88bbcba6e6e951570d462b0c0a231adaea"}
Sep 10 15:20:38.270223 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:38.269922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:38.316630 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:38.316582    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2" podStartSLOduration=1.569869611 podStartE2EDuration="6.316566942s" totalImagesPullingTime="3.703174381s" totalInitContainerRuntime="1.04352295s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:20:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:20:38.31528427 +0000 UTC m=+278.851239843" watchObservedRunningTime="2026-09-10 15:20:38.316566942 +0000 UTC m=+278.852522505"
Sep 10 15:20:39.900916 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:39.900885    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:39.979155 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:39.979122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:49.276733 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:49.276698    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76f847f9cd-lz6z2"
Sep 10 15:20:50.215810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.215771    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"]
Sep 10 15:20:50.229393 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.229361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.232036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.232016    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bfnmr\""
Sep 10 15:20:50.233115 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.233094    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 15:20:50.233218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.233105    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 15:20:50.233218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.233141    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 15:20:50.233218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.233146    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:20:50.233557 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.233541    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 15:20:50.238423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.238398    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-cert\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.238548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.238518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-metrics-cert\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.238634 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.238604    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-manager-config\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.238693 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.238673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnkdh\" (UniqueName: \"kubernetes.io/projected/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-kube-api-access-hnkdh\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.240074 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.240055    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"]
Sep 10 15:20:50.339820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.339779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnkdh\" (UniqueName: \"kubernetes.io/projected/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-kube-api-access-hnkdh\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.340269 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.339830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-cert\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.340269 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.339876    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-metrics-cert\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.340269 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.339965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-manager-config\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.340624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.340606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-manager-config\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.342341 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.342317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-metrics-cert\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.342485 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.342442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-cert\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.349482 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.349447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnkdh\" (UniqueName: \"kubernetes.io/projected/9f20cc7e-4737-4ce1-86a2-cba89e5e70b0-kube-api-access-hnkdh\") pod \"lws-controller-manager-578d56c89f-r248s\" (UID: \"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0\") " pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.539623 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.539531    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:50.816747 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:50.816717    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"]
Sep 10 15:20:50.818474 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:20:50.818436    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f20cc7e_4737_4ce1_86a2_cba89e5e70b0.slice/crio-e9e45cc053e00768bb5a43d5756b79f5f608f9f4cc277417b65614605b849f3b WatchSource:0}: Error finding container e9e45cc053e00768bb5a43d5756b79f5f608f9f4cc277417b65614605b849f3b: Status 404 returned error can't find the container with id e9e45cc053e00768bb5a43d5756b79f5f608f9f4cc277417b65614605b849f3b
Sep 10 15:20:51.335620 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:51.335584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s" event={"ID":"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0","Type":"ContainerStarted","Data":"e9e45cc053e00768bb5a43d5756b79f5f608f9f4cc277417b65614605b849f3b"}
Sep 10 15:20:51.637316 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:51.637209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:51.778799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:51.778755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:51.976810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:51.976659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:52.167974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.167930    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"]
Sep 10 15:20:52.171952 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.171926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.185384 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.185358    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 15:20:52.186055 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.186036    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 15:20:52.186055 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.186051    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 15:20:52.191522 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.191501    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 15:20:52.191900 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.191883    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7fz72\""
Sep 10 15:20:52.223432 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.223388    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"]
Sep 10 15:20:52.257953 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.257865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:52.260097 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.260056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgbn\" (UniqueName: \"kubernetes.io/projected/bde8662d-29b3-4cfe-aed1-bacf36903c57-kube-api-access-xwgbn\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.260242 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.260144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bde8662d-29b3-4cfe-aed1-bacf36903c57-tls-certs\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.260242 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.260230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bde8662d-29b3-4cfe-aed1-bacf36903c57-tmp\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.361003 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.360961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bde8662d-29b3-4cfe-aed1-bacf36903c57-tls-certs\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.361181 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.361041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bde8662d-29b3-4cfe-aed1-bacf36903c57-tmp\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.361181 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.361109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgbn\" (UniqueName: \"kubernetes.io/projected/bde8662d-29b3-4cfe-aed1-bacf36903c57-kube-api-access-xwgbn\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.363561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.363533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bde8662d-29b3-4cfe-aed1-bacf36903c57-tmp\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.363764 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.363743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bde8662d-29b3-4cfe-aed1-bacf36903c57-tls-certs\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.406227 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.406189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgbn\" (UniqueName: \"kubernetes.io/projected/bde8662d-29b3-4cfe-aed1-bacf36903c57-kube-api-access-xwgbn\") pod \"kube-auth-proxy-84bd548dcf-zczq4\" (UID: \"bde8662d-29b3-4cfe-aed1-bacf36903c57\") " pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:52.484576 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:52.484539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"
Sep 10 15:20:53.601329 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:53.601304    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4"]
Sep 10 15:20:53.603409 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:20:53.603382    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde8662d_29b3_4cfe_aed1_bacf36903c57.slice/crio-a3175c294ef6b63c9950875768bfbf388bceb2131c5e023f5e5b51a979189dd5 WatchSource:0}: Error finding container a3175c294ef6b63c9950875768bfbf388bceb2131c5e023f5e5b51a979189dd5: Status 404 returned error can't find the container with id a3175c294ef6b63c9950875768bfbf388bceb2131c5e023f5e5b51a979189dd5
Sep 10 15:20:53.926674 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:53.926582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:54.026485 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:54.026433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:54.357418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:54.357276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4" event={"ID":"bde8662d-29b3-4cfe-aed1-bacf36903c57","Type":"ContainerStarted","Data":"a3175c294ef6b63c9950875768bfbf388bceb2131c5e023f5e5b51a979189dd5"}
Sep 10 15:20:54.365578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:54.363980    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s" event={"ID":"9f20cc7e-4737-4ce1-86a2-cba89e5e70b0","Type":"ContainerStarted","Data":"b0980cd267dc19d5787ee78f89bc443f41e402d5e9ec868fc7531597d1b8454c"}
Sep 10 15:20:54.365578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:54.364918    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:20:54.387128 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:54.385998    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s" podStartSLOduration=1.807178096 podStartE2EDuration="4.385975707s" totalImagesPullingTime="2.578797611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:20:54.385275183 +0000 UTC m=+294.921230774" watchObservedRunningTime="2026-09-10 15:20:54.385975707 +0000 UTC m=+294.921931271"
Sep 10 15:20:55.128735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:55.128693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:56.374955 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:56.374916    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4" event={"ID":"bde8662d-29b3-4cfe-aed1-bacf36903c57","Type":"ContainerStarted","Data":"cecd36ec8da896df3c4d7f1f8d3e4335665b57820fbef6248bc6475d968c55a0"}
Sep 10 15:20:56.393420 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:56.393374    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84bd548dcf-zczq4" podStartSLOduration=1.8216384350000001 podStartE2EDuration="4.393359853s" totalImagesPullingTime="2.571721418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:20:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:20:56.392972942 +0000 UTC m=+296.928928529" watchObservedRunningTime="2026-09-10 15:20:56.393359853 +0000 UTC m=+296.929315415"
Sep 10 15:20:58.028120 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:58.028085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:58.086344 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:58.086316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:58.803626 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:58.803596    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:58.999726 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:58.999695    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:59.146694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:59.146661    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:20:59.854438 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:59.854411    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:20:59.854937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:59.854911    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:20:59.859809 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:20:59.859792    2562 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 15:21:02.941187 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:02.941146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:05.526604 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:05.526559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:06.377314 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:06.377275    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-578d56c89f-r248s"
Sep 10 15:21:14.367089 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:14.367045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:14.493141 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:14.493080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:14.597165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:14.597123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:14.923189 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:14.923153    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:15.747054 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:15.747012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:15.846312 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:15.846279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:15.960042 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:15.960013    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:16.110702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:16.110664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:41.706469 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:41.706419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:41.802885 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:41.802851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:43.469866 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:43.469834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:21:43.569431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:21:43.569400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:03.767776 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:03.767701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:03.866954 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:03.866915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:16.120615 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:16.120582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:16.944990 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:16.944930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:20.142810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:20.142772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:20.358232 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:20.358197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:20.461484 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:20.461356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:21.817758 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:21.817717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:22.316829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:22.316787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:22.709346 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:22.709250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:23.109662 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:23.109628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:23.446003 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:23.445914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:23.544191 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:23.544152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:26.526699 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:26.526667    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:27.969304 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:27.969270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:29.519528 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:29.519496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:29.634396 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:29.634356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:29.915972 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:29.915937    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:30.013480 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:30.013417    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:33.416246 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.416204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:33.422859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.422831    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"]
Sep 10 15:22:33.425157 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.425138    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.427705 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.427676    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 15:22:33.427824 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.427677    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 15:22:33.427824 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.427719    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 15:22:33.427824 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.427733    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h5hpc\""
Sep 10 15:22:33.428566 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.428553    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 15:22:33.433381 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.433358    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"]
Sep 10 15:22:33.505802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.505760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4cf61341-b3fd-4db1-88b2-36909e556d7b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.505996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.505809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bp8t\" (UniqueName: \"kubernetes.io/projected/4cf61341-b3fd-4db1-88b2-36909e556d7b-kube-api-access-4bp8t\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.505996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.505925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cf61341-b3fd-4db1-88b2-36909e556d7b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.516012 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.515978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:33.607433 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.607380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bp8t\" (UniqueName: \"kubernetes.io/projected/4cf61341-b3fd-4db1-88b2-36909e556d7b-kube-api-access-4bp8t\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.607659 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.607531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cf61341-b3fd-4db1-88b2-36909e556d7b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.607659 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.607603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4cf61341-b3fd-4db1-88b2-36909e556d7b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.608245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.608219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4cf61341-b3fd-4db1-88b2-36909e556d7b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.610053 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.610030    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cf61341-b3fd-4db1-88b2-36909e556d7b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.616491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.616462    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:33.617759 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.617733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bp8t\" (UniqueName: \"kubernetes.io/projected/4cf61341-b3fd-4db1-88b2-36909e556d7b-kube-api-access-4bp8t\") pod \"kuadrant-console-plugin-fbdd4f445-sdxdh\" (UID: \"4cf61341-b3fd-4db1-88b2-36909e556d7b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:33.745695 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:33.745592    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"
Sep 10 15:22:34.028029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.027934    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh"]
Sep 10 15:22:34.040435 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:22:34.040401    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cf61341_b3fd_4db1_88b2_36909e556d7b.slice/crio-e4515a57d125a7bc14050895dfbff7ab2624385bf944d4871cc21144979accab WatchSource:0}: Error finding container e4515a57d125a7bc14050895dfbff7ab2624385bf944d4871cc21144979accab: Status 404 returned error can't find the container with id e4515a57d125a7bc14050895dfbff7ab2624385bf944d4871cc21144979accab
Sep 10 15:22:34.042042 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.042026    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:22:34.136937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.136905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:34.261505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.261440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:34.361772 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.361730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:34.819428 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.819395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh" event={"ID":"4cf61341-b3fd-4db1-88b2-36909e556d7b","Type":"ContainerStarted","Data":"e4515a57d125a7bc14050895dfbff7ab2624385bf944d4871cc21144979accab"}
Sep 10 15:22:34.872039 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.872001    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:34.967380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:34.967341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:38.842197 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:38.842153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh" event={"ID":"4cf61341-b3fd-4db1-88b2-36909e556d7b","Type":"ContainerStarted","Data":"8e8fd74bdf957199f381ca79ea46f4f39a44f43b6bce6bbe86ac86c807714e27"}
Sep 10 15:22:38.863130 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:38.863072    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-sdxdh" podStartSLOduration=1.258510133 podStartE2EDuration="5.863054187s" totalImagesPullingTime="4.604544054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:22:38.861108257 +0000 UTC m=+399.397063823" watchObservedRunningTime="2026-09-10 15:22:38.863054187 +0000 UTC m=+399.399009748"
Sep 10 15:22:39.094189 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:39.094097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:39.221054 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:39.220977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:39.296956 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:39.296915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:44.623396 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:44.623355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:44.737135 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:44.737095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:44.827469 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:44.827409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:46.141634 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:46.141587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:46.238160 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:46.238125    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:50.169006 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:50.168951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:50.271033 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:50.270999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:50.732247 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:50.732212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:50.833683 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:50.833648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:53.740297 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:53.740254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:53.813294 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:53.813260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:59.104612 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:59.104570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:59.206778 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:59.206740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:22:59.303330 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:22:59.303299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:06.137650 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:06.137574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:06.239812 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:06.239773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:13.585907 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:13.585865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:13.646392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:13.646357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:14.224889 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:14.224851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:14.733327 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:14.733292    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:23.666431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:23.666335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:30.229143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.229103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:30.335053 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.335021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:30.395245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.395207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:30.638121 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.638083    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:23:30.640427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.640380    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:30.643052 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.643031    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 15:23:30.651048 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.651019    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:23:30.682230 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.682205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:23:30.738092 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.738058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7b3a37c-4776-4f5e-9df9-e1e62928f687-config-file\") pod \"limitador-limitador-84994f7ffc-xvjzr\" (UID: \"a7b3a37c-4776-4f5e-9df9-e1e62928f687\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:30.738229 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.738099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcck7\" (UniqueName: \"kubernetes.io/projected/a7b3a37c-4776-4f5e-9df9-e1e62928f687-kube-api-access-rcck7\") pod \"limitador-limitador-84994f7ffc-xvjzr\" (UID: \"a7b3a37c-4776-4f5e-9df9-e1e62928f687\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:30.839068 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.839040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7b3a37c-4776-4f5e-9df9-e1e62928f687-config-file\") pod \"limitador-limitador-84994f7ffc-xvjzr\" (UID: \"a7b3a37c-4776-4f5e-9df9-e1e62928f687\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:30.839221 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.839073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcck7\" (UniqueName: \"kubernetes.io/projected/a7b3a37c-4776-4f5e-9df9-e1e62928f687-kube-api-access-rcck7\") pod \"limitador-limitador-84994f7ffc-xvjzr\" (UID: \"a7b3a37c-4776-4f5e-9df9-e1e62928f687\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:30.839717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.839691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7b3a37c-4776-4f5e-9df9-e1e62928f687-config-file\") pod \"limitador-limitador-84994f7ffc-xvjzr\" (UID: \"a7b3a37c-4776-4f5e-9df9-e1e62928f687\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:30.847749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.847729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcck7\" (UniqueName: \"kubernetes.io/projected/a7b3a37c-4776-4f5e-9df9-e1e62928f687-kube-api-access-rcck7\") pod \"limitador-limitador-84994f7ffc-xvjzr\" (UID: \"a7b3a37c-4776-4f5e-9df9-e1e62928f687\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:30.952291 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:30.952202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:31.087921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:31.087882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:31.172296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:31.172262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:31.220226 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:31.220199    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:23:31.222266 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:23:31.222240    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b3a37c_4776_4f5e_9df9_e1e62928f687.slice/crio-e0374301866d231ca6d7e910ba24338a2bad83bc40f6ac74fed8a5ec0cfc266b WatchSource:0}: Error finding container e0374301866d231ca6d7e910ba24338a2bad83bc40f6ac74fed8a5ec0cfc266b: Status 404 returned error can't find the container with id e0374301866d231ca6d7e910ba24338a2bad83bc40f6ac74fed8a5ec0cfc266b
Sep 10 15:23:32.085323 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:32.085288    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr" event={"ID":"a7b3a37c-4776-4f5e-9df9-e1e62928f687","Type":"ContainerStarted","Data":"e0374301866d231ca6d7e910ba24338a2bad83bc40f6ac74fed8a5ec0cfc266b"}
Sep 10 15:23:34.005388 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:34.005316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:34.197275 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:34.197240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:38.120549 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:38.120492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr" event={"ID":"a7b3a37c-4776-4f5e-9df9-e1e62928f687","Type":"ContainerStarted","Data":"f2a10c629511fb9a722e35e11fa0d96a66226d7dbdb986d815eff3a21229ca3d"}
Sep 10 15:23:38.120992 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:38.120611    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:38.140334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:38.140290    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr" podStartSLOduration=2.297433671 podStartE2EDuration="8.140272788s" totalImagesPullingTime="5.842839117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:23:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:23:38.139518443 +0000 UTC m=+458.675474015" watchObservedRunningTime="2026-09-10 15:23:38.140272788 +0000 UTC m=+458.676228349"
Sep 10 15:23:49.127399 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:49.127361    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"
Sep 10 15:23:53.169166 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:53.169129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:53.278355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:53.278324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:23:53.643609 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:23:53.643570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:05.612976 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:05.612935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:06.493922 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:06.493886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:06.605186 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:06.605152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:13.866292 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.866243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:13.893500 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.893442    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-588774b897-pv6cd"]
Sep 10 15:24:13.897618 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.897596    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:13.901404 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.901380    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 15:24:13.901549 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.901415    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 15:24:13.901549 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.901380    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 15:24:13.901549 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.901384    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-gqnvd\""
Sep 10 15:24:13.907497 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.907474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588774b897-pv6cd"]
Sep 10 15:24:13.984871 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:13.984837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:14.025712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.025667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-webhook-cert\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.025712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.025714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.025969 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.025802    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrpwp\" (UniqueName: \"kubernetes.io/projected/38dcd13a-6cbf-4332-b44d-234b034d907b-kube-api-access-wrpwp\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.083404 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.083370    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:14.127042 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.126943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-webhook-cert\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.127042 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.126981    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.127042 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.127023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrpwp\" (UniqueName: \"kubernetes.io/projected/38dcd13a-6cbf-4332-b44d-234b034d907b-kube-api-access-wrpwp\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.127342 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:24:14.127153    2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 15:24:14.127342 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:24:14.127215    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs podName:38dcd13a-6cbf-4332-b44d-234b034d907b nodeName:}" failed. No retries permitted until 2026-09-10 15:24:14.627193949 +0000 UTC m=+495.163149503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs") pod "maas-controller-588774b897-pv6cd" (UID: "38dcd13a-6cbf-4332-b44d-234b034d907b") : secret "maas-controller-metrics-tls" not found
Sep 10 15:24:14.129563 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.129538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-webhook-cert\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.141514 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.141478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrpwp\" (UniqueName: \"kubernetes.io/projected/38dcd13a-6cbf-4332-b44d-234b034d907b-kube-api-access-wrpwp\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.589921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.589892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:14.632757 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.632717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.635188 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.635159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs\") pod \"maas-controller-588774b897-pv6cd\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") " pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:14.809418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:14.809371    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:15.081247 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:15.081225    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588774b897-pv6cd"]
Sep 10 15:24:15.083915 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:24:15.083886    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dcd13a_6cbf_4332_b44d_234b034d907b.slice/crio-7cb21037540c4beb31816de9c3522c2ce157b0221643be45f71d1a61d2f6d61b WatchSource:0}: Error finding container 7cb21037540c4beb31816de9c3522c2ce157b0221643be45f71d1a61d2f6d61b: Status 404 returned error can't find the container with id 7cb21037540c4beb31816de9c3522c2ce157b0221643be45f71d1a61d2f6d61b
Sep 10 15:24:15.306507 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:15.306394    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588774b897-pv6cd" event={"ID":"38dcd13a-6cbf-4332-b44d-234b034d907b","Type":"ContainerStarted","Data":"7cb21037540c4beb31816de9c3522c2ce157b0221643be45f71d1a61d2f6d61b"}
Sep 10 15:24:17.734413 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:17.734360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:17.834812 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:17.834762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:18.324891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:18.324859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588774b897-pv6cd" event={"ID":"38dcd13a-6cbf-4332-b44d-234b034d907b","Type":"ContainerStarted","Data":"70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba"}
Sep 10 15:24:18.325070 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:18.324961    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:18.348606 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:18.348541    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-588774b897-pv6cd" podStartSLOduration=2.875203092 podStartE2EDuration="5.348524753s" totalImagesPullingTime="2.473321661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:24:18.347015718 +0000 UTC m=+498.882971305" watchObservedRunningTime="2026-09-10 15:24:18.348524753 +0000 UTC m=+498.884480315"
Sep 10 15:24:29.309833 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.309795    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588774b897-pv6cd"]
Sep 10 15:24:29.310294 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.310023    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-588774b897-pv6cd" podUID="38dcd13a-6cbf-4332-b44d-234b034d907b" containerName="manager" containerID="cri-o://70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba" gracePeriod=30
Sep 10 15:24:29.311836 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.311809    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:29.555672 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.555642    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:29.667975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.667929    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrpwp\" (UniqueName: \"kubernetes.io/projected/38dcd13a-6cbf-4332-b44d-234b034d907b-kube-api-access-wrpwp\") pod \"38dcd13a-6cbf-4332-b44d-234b034d907b\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") "
Sep 10 15:24:29.667975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.667981    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs\") pod \"38dcd13a-6cbf-4332-b44d-234b034d907b\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") "
Sep 10 15:24:29.668218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.668024    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-webhook-cert\") pod \"38dcd13a-6cbf-4332-b44d-234b034d907b\" (UID: \"38dcd13a-6cbf-4332-b44d-234b034d907b\") "
Sep 10 15:24:29.670235 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.670207    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "38dcd13a-6cbf-4332-b44d-234b034d907b" (UID: "38dcd13a-6cbf-4332-b44d-234b034d907b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:24:29.670326 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.670227    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38dcd13a-6cbf-4332-b44d-234b034d907b-kube-api-access-wrpwp" (OuterVolumeSpecName: "kube-api-access-wrpwp") pod "38dcd13a-6cbf-4332-b44d-234b034d907b" (UID: "38dcd13a-6cbf-4332-b44d-234b034d907b"). InnerVolumeSpecName "kube-api-access-wrpwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:24:29.670326 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.670231    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "38dcd13a-6cbf-4332-b44d-234b034d907b" (UID: "38dcd13a-6cbf-4332-b44d-234b034d907b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:24:29.768892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.768852    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrpwp\" (UniqueName: \"kubernetes.io/projected/38dcd13a-6cbf-4332-b44d-234b034d907b-kube-api-access-wrpwp\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:24:29.768892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.768885    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-metrics-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:24:29.768892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:29.768896    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38dcd13a-6cbf-4332-b44d-234b034d907b-webhook-cert\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:24:30.388011 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.387973    2562 generic.go:356] "Generic (PLEG): container finished" podID="38dcd13a-6cbf-4332-b44d-234b034d907b" containerID="70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba" exitCode=0
Sep 10 15:24:30.388414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.388038    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588774b897-pv6cd"
Sep 10 15:24:30.388414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.388059    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588774b897-pv6cd" event={"ID":"38dcd13a-6cbf-4332-b44d-234b034d907b","Type":"ContainerDied","Data":"70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba"}
Sep 10 15:24:30.388414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.388115    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588774b897-pv6cd" event={"ID":"38dcd13a-6cbf-4332-b44d-234b034d907b","Type":"ContainerDied","Data":"7cb21037540c4beb31816de9c3522c2ce157b0221643be45f71d1a61d2f6d61b"}
Sep 10 15:24:30.388414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.388138    2562 scope.go:117] "RemoveContainer" containerID="70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba"
Sep 10 15:24:30.396709 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.396687    2562 scope.go:117] "RemoveContainer" containerID="70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba"
Sep 10 15:24:30.397048 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:24:30.397030    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba\": container with ID starting with 70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba not found: ID does not exist" containerID="70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba"
Sep 10 15:24:30.397100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.397058    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba"} err="failed to get container status \"70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba\": rpc error: code = NotFound desc = could not find container \"70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba\": container with ID starting with 70dade078152e0b56d61715d3fb9a0fc1f21782d75f45e5bf40e49da62217cba not found: ID does not exist"
Sep 10 15:24:30.404674 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.404648    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588774b897-pv6cd"]
Sep 10 15:24:30.408204 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:30.408180    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-588774b897-pv6cd"]
Sep 10 15:24:31.954922 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:31.954888    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38dcd13a-6cbf-4332-b44d-234b034d907b" path="/var/lib/kubelet/pods/38dcd13a-6cbf-4332-b44d-234b034d907b/volumes"
Sep 10 15:24:49.505904 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:49.505863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:49.586495 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:49.586437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:51.052567 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.052500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:51.153599 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.153557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:51.241403 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.241365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:51.587348 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.587310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:51.685932 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.685894    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:51.769108 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.769072    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"]
Sep 10 15:24:51.769543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.769529    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38dcd13a-6cbf-4332-b44d-234b034d907b" containerName="manager"
Sep 10 15:24:51.769597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.769545    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="38dcd13a-6cbf-4332-b44d-234b034d907b" containerName="manager"
Sep 10 15:24:51.769636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.769615    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="38dcd13a-6cbf-4332-b44d-234b034d907b" containerName="manager"
Sep 10 15:24:51.777809 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.777785    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:51.780994 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.780971    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 15:24:51.781334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.781317    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xl6p5\""
Sep 10 15:24:51.787143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.787115    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"]
Sep 10 15:24:51.845331 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.845242    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:24:51.852310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.852280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:24:51.872286 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.872255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9zrp\" (UniqueName: \"kubernetes.io/projected/db0c1425-6b28-4af5-8a6f-04ef280d2ddb-kube-api-access-z9zrp\") pod \"payload-pre-processing-59fbc47bf8-fldkc\" (UID: \"db0c1425-6b28-4af5-8a6f-04ef280d2ddb\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:51.872437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.872371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0c1425-6b28-4af5-8a6f-04ef280d2ddb-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fldkc\" (UID: \"db0c1425-6b28-4af5-8a6f-04ef280d2ddb\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:51.973602 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.973560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0c1425-6b28-4af5-8a6f-04ef280d2ddb-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fldkc\" (UID: \"db0c1425-6b28-4af5-8a6f-04ef280d2ddb\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:51.973775 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.973675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9zrp\" (UniqueName: \"kubernetes.io/projected/db0c1425-6b28-4af5-8a6f-04ef280d2ddb-kube-api-access-z9zrp\") pod \"payload-pre-processing-59fbc47bf8-fldkc\" (UID: \"db0c1425-6b28-4af5-8a6f-04ef280d2ddb\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:51.974375 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.974355    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0c1425-6b28-4af5-8a6f-04ef280d2ddb-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fldkc\" (UID: \"db0c1425-6b28-4af5-8a6f-04ef280d2ddb\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:51.995195 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:51.995159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9zrp\" (UniqueName: \"kubernetes.io/projected/db0c1425-6b28-4af5-8a6f-04ef280d2ddb-kube-api-access-z9zrp\") pod \"payload-pre-processing-59fbc47bf8-fldkc\" (UID: \"db0c1425-6b28-4af5-8a6f-04ef280d2ddb\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:52.090507 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:52.090409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:52.366484 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:52.366437    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"]
Sep 10 15:24:52.368541 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:24:52.368511    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0c1425_6b28_4af5_8a6f_04ef280d2ddb.slice/crio-01b170eca788df27c2ae8a150968833f749fa54fefa930c8e8a1ad6403279ab9 WatchSource:0}: Error finding container 01b170eca788df27c2ae8a150968833f749fa54fefa930c8e8a1ad6403279ab9: Status 404 returned error can't find the container with id 01b170eca788df27c2ae8a150968833f749fa54fefa930c8e8a1ad6403279ab9
Sep 10 15:24:52.500008 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:52.499972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc" event={"ID":"db0c1425-6b28-4af5-8a6f-04ef280d2ddb","Type":"ContainerStarted","Data":"01b170eca788df27c2ae8a150968833f749fa54fefa930c8e8a1ad6403279ab9"}
Sep 10 15:24:55.516535 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:55.516492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc" event={"ID":"db0c1425-6b28-4af5-8a6f-04ef280d2ddb","Type":"ContainerStarted","Data":"7be6636fd74ae27fcea1c379f250af776802c866bc38e70bf27e1cb0550348db"}
Sep 10 15:24:55.516913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:55.516598    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:24:55.539213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:24:55.539162    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc" podStartSLOduration=2.293590287 podStartE2EDuration="4.539146732s" totalImagesPullingTime="2.245556445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:24:55.538347799 +0000 UTC m=+536.074303381" watchObservedRunningTime="2026-09-10 15:24:55.539146732 +0000 UTC m=+536.075102291"
Sep 10 15:25:02.444068 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:02.444035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:02.556670 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:02.556632    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:06.523818 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:06.523784    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fldkc"
Sep 10 15:25:15.135771 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.135728    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"]
Sep 10 15:25:15.140284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.140265    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.145225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.145201    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 15:25:15.145355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.145201    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 15:25:15.145355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.145327    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8dhf4\""
Sep 10 15:25:15.145493 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.145330    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 15:25:15.161032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.161004    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"]
Sep 10 15:25:15.165405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.165380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:15.172677 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.172654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfk4z\" (UniqueName: \"kubernetes.io/projected/27b680e6-7512-42e1-a898-6a621acf3821-kube-api-access-vfk4z\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.172747 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.172726    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.172802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.172763    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.172840 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.172801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.172887 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.172839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27b680e6-7512-42e1-a898-6a621acf3821-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.172940 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.172913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.273684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.273642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfk4z\" (UniqueName: \"kubernetes.io/projected/27b680e6-7512-42e1-a898-6a621acf3821-kube-api-access-vfk4z\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.273888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.273708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.273888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.273740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.273888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.273764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.273888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.273801    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27b680e6-7512-42e1-a898-6a621acf3821-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.273888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.273839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.274242 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.274209    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.274356 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.274221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.274356 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.274276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.275944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.275926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27b680e6-7512-42e1-a898-6a621acf3821-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.276291 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.276275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27b680e6-7512-42e1-a898-6a621acf3821-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.288964 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.288936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfk4z\" (UniqueName: \"kubernetes.io/projected/27b680e6-7512-42e1-a898-6a621acf3821-kube-api-access-vfk4z\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9\" (UID: \"27b680e6-7512-42e1-a898-6a621acf3821\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.450312 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.450215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:15.737808 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.737747    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"]
Sep 10 15:25:15.740046 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:25:15.740022    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b680e6_7512_42e1_a898_6a621acf3821.slice/crio-2ba90c9500865a6101a27655d9ce66e18a2d32bb0317ace06b08fcca368c11b8 WatchSource:0}: Error finding container 2ba90c9500865a6101a27655d9ce66e18a2d32bb0317ace06b08fcca368c11b8: Status 404 returned error can't find the container with id 2ba90c9500865a6101a27655d9ce66e18a2d32bb0317ace06b08fcca368c11b8
Sep 10 15:25:15.838164 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:15.838121    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:25:16.624843 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:16.624806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9" event={"ID":"27b680e6-7512-42e1-a898-6a621acf3821","Type":"ContainerStarted","Data":"2ba90c9500865a6101a27655d9ce66e18a2d32bb0317ace06b08fcca368c11b8"}
Sep 10 15:25:17.630853 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:17.630819    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9" event={"ID":"27b680e6-7512-42e1-a898-6a621acf3821","Type":"ContainerStarted","Data":"107709a1f38e31004c3952184f4581f58b1526f56836f0a525415ef71768b85d"}
Sep 10 15:25:17.650322 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:17.650274    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9" podStartSLOduration=1.475854253 podStartE2EDuration="2.650258091s" totalImagesPullingTime="1.174403838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:25:17.648687496 +0000 UTC m=+558.184643075" watchObservedRunningTime="2026-09-10 15:25:17.650258091 +0000 UTC m=+558.186213653"
Sep 10 15:25:24.193306 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:24.193267    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:24.323074 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:24.323035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:24.850227 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:24.850190    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:25:25.450808 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:25.450772    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:25.451204 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:25.450835    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:25.463755 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:25.463717    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:25.679642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:25.679600    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9"
Sep 10 15:25:38.436133 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:38.436099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:39.430162 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:39.430122    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:25:41.443357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:41.443318    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:43.131868 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:43.131820    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:25:48.820092 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:48.820047    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:48.906585 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:48.906548    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:50.342010 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.341968    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"]
Sep 10 15:25:50.351941 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.351908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.354774 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.354750    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 15:25:50.357574 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.357507    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"]
Sep 10 15:25:50.376395 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.376360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:50.413710 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.413671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.413710 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.413708    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cc1af06-6148-4c96-b823-2a31315ff64e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.413913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.413809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.413913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.413850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.413913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.413894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8jw2\" (UniqueName: \"kubernetes.io/projected/6cc1af06-6148-4c96-b823-2a31315ff64e-kube-api-access-p8jw2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.414013 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.413933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515153 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8jw2\" (UniqueName: \"kubernetes.io/projected/6cc1af06-6148-4c96-b823-2a31315ff64e-kube-api-access-p8jw2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515180    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cc1af06-6148-4c96-b823-2a31315ff64e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515648 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515788 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.515788 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.515780    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.517438 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.517415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cc1af06-6148-4c96-b823-2a31315ff64e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.517766 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.517740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cc1af06-6148-4c96-b823-2a31315ff64e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.524420 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.524394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8jw2\" (UniqueName: \"kubernetes.io/projected/6cc1af06-6148-4c96-b823-2a31315ff64e-kube-api-access-p8jw2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r\" (UID: \"6cc1af06-6148-4c96-b823-2a31315ff64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.663620 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.663525    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:25:50.943583 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:50.943491    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"]
Sep 10 15:25:50.947236 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:25:50.947205    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc1af06_6148_4c96_b823_2a31315ff64e.slice/crio-c2fa7e7f25969d5d62a3fc5f9eef2bb2207f0a1eb9747db3595c300863447f32 WatchSource:0}: Error finding container c2fa7e7f25969d5d62a3fc5f9eef2bb2207f0a1eb9747db3595c300863447f32: Status 404 returned error can't find the container with id c2fa7e7f25969d5d62a3fc5f9eef2bb2207f0a1eb9747db3595c300863447f32
Sep 10 15:25:51.803287 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:51.803247    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r" event={"ID":"6cc1af06-6148-4c96-b823-2a31315ff64e","Type":"ContainerStarted","Data":"fb4941cc266bf795b68443fb6d76f5002ffa67cd5a87642873e31cee9c9d9c86"}
Sep 10 15:25:51.803806 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:51.803308    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r" event={"ID":"6cc1af06-6148-4c96-b823-2a31315ff64e","Type":"ContainerStarted","Data":"c2fa7e7f25969d5d62a3fc5f9eef2bb2207f0a1eb9747db3595c300863447f32"}
Sep 10 15:25:51.824138 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:51.824082    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r" podStartSLOduration=1.59862063 podStartE2EDuration="1.824065357s" totalImagesPullingTime="225.444727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:25:51.823078077 +0000 UTC m=+592.359033701" watchObservedRunningTime="2026-09-10 15:25:51.824065357 +0000 UTC m=+592.360020919"
Sep 10 15:25:51.932344 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:51.932304    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:25:57.939559 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.939520    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"]
Sep 10 15:25:57.943154 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.943129    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:57.946025 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.946003    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 15:25:57.957569 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.957543    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"]
Sep 10 15:25:57.975218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.975158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:25:57.984417 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.984397    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57e8df30-fda5-447c-96cb-edda7fe425f0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:57.984540 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.984423    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:57.984540 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.984443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:57.984540 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.984480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:57.984645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.984564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqlv6\" (UniqueName: \"kubernetes.io/projected/57e8df30-fda5-447c-96cb-edda7fe425f0-kube-api-access-dqlv6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:57.984645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:57.984596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086062 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqlv6\" (UniqueName: \"kubernetes.io/projected/57e8df30-fda5-447c-96cb-edda7fe425f0-kube-api-access-dqlv6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086062 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57e8df30-fda5-447c-96cb-edda7fe425f0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086641 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086713 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086623    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.086713 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.086668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.088485 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.088444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57e8df30-fda5-447c-96cb-edda7fe425f0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.088768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.088746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57e8df30-fda5-447c-96cb-edda7fe425f0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.094752 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.094730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqlv6\" (UniqueName: \"kubernetes.io/projected/57e8df30-fda5-447c-96cb-edda7fe425f0-kube-api-access-dqlv6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hvbx\" (UID: \"57e8df30-fda5-447c-96cb-edda7fe425f0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.257041 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.256946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:25:58.533765 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.533725    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"]
Sep 10 15:25:58.536724 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:25:58.536695    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e8df30_fda5_447c_96cb_edda7fe425f0.slice/crio-bd335f3c282b1c101840f75479df352b614f362a9d0417a4cea86854c41ce261 WatchSource:0}: Error finding container bd335f3c282b1c101840f75479df352b614f362a9d0417a4cea86854c41ce261: Status 404 returned error can't find the container with id bd335f3c282b1c101840f75479df352b614f362a9d0417a4cea86854c41ce261
Sep 10 15:25:58.838137 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.838091    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx" event={"ID":"57e8df30-fda5-447c-96cb-edda7fe425f0","Type":"ContainerStarted","Data":"cf4ffda172ddbe273769a9973b2c804d0df8aaebad3fc56d89c48723a9c400c1"}
Sep 10 15:25:58.838344 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.838203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx" event={"ID":"57e8df30-fda5-447c-96cb-edda7fe425f0","Type":"ContainerStarted","Data":"bd335f3c282b1c101840f75479df352b614f362a9d0417a4cea86854c41ce261"}
Sep 10 15:25:58.858707 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:58.858652    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx" podStartSLOduration=1.659304978 podStartE2EDuration="1.858635882s" totalImagesPullingTime="199.330904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:25:58.857307455 +0000 UTC m=+599.393263035" watchObservedRunningTime="2026-09-10 15:25:58.858635882 +0000 UTC m=+599.394591444"
Sep 10 15:25:59.832590 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:59.832549    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:25:59.886718 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:59.886688    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:25:59.886937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:25:59.886918    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:26:00.664272 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:00.664228    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:26:00.664534 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:00.664305    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:26:00.676737 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:00.676703    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:26:00.866414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:00.866374    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r"
Sep 10 15:26:02.818958 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.818921    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"]
Sep 10 15:26:02.822881 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.822859    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:02.825379 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.825357    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 15:26:02.833728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.833704    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"]
Sep 10 15:26:02.861309 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.861275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:26:02.940961 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.940905    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:02.940961 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.940963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:02.941245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.940991    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c8ddcb-fccf-4d77-86e1-4fe862067934-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:02.941245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.941038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:02.941245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.941056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:02.941245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:02.941073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n665q\" (UniqueName: \"kubernetes.io/projected/e7c8ddcb-fccf-4d77-86e1-4fe862067934-kube-api-access-n665q\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.041977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n665q\" (UniqueName: \"kubernetes.io/projected/e7c8ddcb-fccf-4d77-86e1-4fe862067934-kube-api-access-n665q\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042241    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c8ddcb-fccf-4d77-86e1-4fe862067934-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042516 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042516 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042482    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.042635 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.042618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.044413 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.044391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c8ddcb-fccf-4d77-86e1-4fe862067934-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.044653 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.044635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c8ddcb-fccf-4d77-86e1-4fe862067934-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.050952 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.050927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n665q\" (UniqueName: \"kubernetes.io/projected/e7c8ddcb-fccf-4d77-86e1-4fe862067934-kube-api-access-n665q\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4\" (UID: \"e7c8ddcb-fccf-4d77-86e1-4fe862067934\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.134561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.134440    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:03.410741 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.410714    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"]
Sep 10 15:26:03.412825 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:26:03.412797    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7c8ddcb_fccf_4d77_86e1_4fe862067934.slice/crio-e5b7ceb6ad34fdba3ef6b0d003d4f93565e5ccfe71a364ca45bcfce9fd7b41ee WatchSource:0}: Error finding container e5b7ceb6ad34fdba3ef6b0d003d4f93565e5ccfe71a364ca45bcfce9fd7b41ee: Status 404 returned error can't find the container with id e5b7ceb6ad34fdba3ef6b0d003d4f93565e5ccfe71a364ca45bcfce9fd7b41ee
Sep 10 15:26:03.870582 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.870492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4" event={"ID":"e7c8ddcb-fccf-4d77-86e1-4fe862067934","Type":"ContainerStarted","Data":"f897062147317747ac8cf972f9f9383feb6d35c46227f2c59bbe1c8a46b0dee4"}
Sep 10 15:26:03.870582 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.870536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4" event={"ID":"e7c8ddcb-fccf-4d77-86e1-4fe862067934","Type":"ContainerStarted","Data":"e5b7ceb6ad34fdba3ef6b0d003d4f93565e5ccfe71a364ca45bcfce9fd7b41ee"}
Sep 10 15:26:03.890966 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:03.890908    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4" podStartSLOduration=1.6865311969999999 podStartE2EDuration="1.890890162s" totalImagesPullingTime="204.358965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:26:03.889652379 +0000 UTC m=+604.425607964" watchObservedRunningTime="2026-09-10 15:26:03.890890162 +0000 UTC m=+604.426845747"
Sep 10 15:26:05.856123 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:05.856080    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:26:08.257942 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:08.257895    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:26:08.258356 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:08.257976    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:26:08.270367 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:08.270334    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:26:08.906624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:08.906583    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hvbx"
Sep 10 15:26:13.135047 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:13.135010    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:13.135501 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:13.135061    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:13.147856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:13.147830    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:26:13.930955 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:26:13.930924    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4"
Sep 10 15:27:11.726435 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:11.726401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:12.841653 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:12.841605    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:27.978463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:27.978410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:28.136794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:28.136729    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:28.360465 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:28.360413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:28.559798 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:28.559761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:29.382537 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:29.382491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:29.482579 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:29.482542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:29.834766 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:29.834712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:31.940791 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:31.940749    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:32.034075 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:32.034041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:32.052617 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:32.052587    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:27:33.260595 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:33.260558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:33.367602 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:33.367565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:33.586949 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:33.586897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:33.953243 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:33.953158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:34.023714 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.023680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:34.065703 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.065665    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"]
Sep 10 15:27:34.069265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.069241    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.072076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.072053    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-229c42\""
Sep 10 15:27:34.081063 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.081035    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"]
Sep 10 15:27:34.145620 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.145575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nt8b\" (UniqueName: \"kubernetes.io/projected/f221ef68-860d-4c17-a111-741551f0c43a-kube-api-access-4nt8b\") pod \"payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.145850 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.145647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f221ef68-860d-4c17-a111-741551f0c43a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.246715 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.246614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f221ef68-860d-4c17-a111-741551f0c43a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.246876 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.246754    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nt8b\" (UniqueName: \"kubernetes.io/projected/f221ef68-860d-4c17-a111-741551f0c43a-kube-api-access-4nt8b\") pod \"payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.247320 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.247300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f221ef68-860d-4c17-a111-741551f0c43a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.256357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.256332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nt8b\" (UniqueName: \"kubernetes.io/projected/f221ef68-860d-4c17-a111-741551f0c43a-kube-api-access-4nt8b\") pod \"payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.380265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.380208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:34.655493 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:34.655440    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"]
Sep 10 15:27:34.658016 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:27:34.657981    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf221ef68_860d_4c17_a111_741551f0c43a.slice/crio-e4fae4747666fef5a3a5fe4f1cbfeae00bf69960a84cafe138ed84739517c50e WatchSource:0}: Error finding container e4fae4747666fef5a3a5fe4f1cbfeae00bf69960a84cafe138ed84739517c50e: Status 404 returned error can't find the container with id e4fae4747666fef5a3a5fe4f1cbfeae00bf69960a84cafe138ed84739517c50e
Sep 10 15:27:35.328053 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:35.328012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7" event={"ID":"f221ef68-860d-4c17-a111-741551f0c43a","Type":"ContainerStarted","Data":"a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e"}
Sep 10 15:27:35.328270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:35.328082    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:35.328270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:35.328103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7" event={"ID":"f221ef68-860d-4c17-a111-741551f0c43a","Type":"ContainerStarted","Data":"e4fae4747666fef5a3a5fe4f1cbfeae00bf69960a84cafe138ed84739517c50e"}
Sep 10 15:27:35.346637 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:35.346587    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7" podStartSLOduration=1.346570761 podStartE2EDuration="1.346570761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:35.345248847 +0000 UTC m=+695.881204420" watchObservedRunningTime="2026-09-10 15:27:35.346570761 +0000 UTC m=+695.882526324"
Sep 10 15:27:36.378069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:36.378039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.531336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:36.531303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.635996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:36.635906    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.732739 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:36.732702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.832318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:36.832279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.935600 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:36.935491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:37.047109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:37.047079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:37.270700 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:37.270610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:39.366106 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:39.366076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:39.460702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:39.460648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:39.931372 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:39.931339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:40.044431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:40.044390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:40.799632 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:40.799595    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:40.860253 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:40.860218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:41.286676 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.286642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:41.471634 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.471601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:41.565749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.565715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:41.740141 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.740103    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"]
Sep 10 15:27:41.743871 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.743846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:41.746974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.746947    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8b74e71b\""
Sep 10 15:27:41.747122 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.747071    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8b74e71b-dockercfg-c9jgw\""
Sep 10 15:27:41.756484 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.756446    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"]
Sep 10 15:27:41.823628 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.823531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z52g\" (UniqueName: \"kubernetes.io/projected/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-kube-api-access-4z52g\") pod \"payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") " pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:41.823628 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.823596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-plugins-config-volume\") pod \"payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") " pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:41.925066 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.925027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z52g\" (UniqueName: \"kubernetes.io/projected/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-kube-api-access-4z52g\") pod \"payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") " pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:41.925281 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.925083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-plugins-config-volume\") pod \"payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") " pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:41.925785 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.925764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-plugins-config-volume\") pod \"payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") " pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:41.947100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:41.947075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z52g\" (UniqueName: \"kubernetes.io/projected/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-kube-api-access-4z52g\") pod \"payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") " pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:42.054288 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.054256    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:42.122767 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.122734    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:42.218751 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.218710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:42.344075 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.344054    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"]
Sep 10 15:27:42.346501 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:27:42.346474    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ab0cec_1730_4eb2_aa15_978b80bcaaa8.slice/crio-5525bcfa5a018342af06c6f43126164f0d65f9fc478b1c0e71dc079823529698 WatchSource:0}: Error finding container 5525bcfa5a018342af06c6f43126164f0d65f9fc478b1c0e71dc079823529698: Status 404 returned error can't find the container with id 5525bcfa5a018342af06c6f43126164f0d65f9fc478b1c0e71dc079823529698
Sep 10 15:27:42.362183 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.362156    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf" event={"ID":"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8","Type":"ContainerStarted","Data":"5525bcfa5a018342af06c6f43126164f0d65f9fc478b1c0e71dc079823529698"}
Sep 10 15:27:42.363995 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.363941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:42.735469 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.735369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:42.835419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.835381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:42.914850 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.914810    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"]
Sep 10 15:27:42.918532 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.918513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:42.921586 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.921562    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-2e2b69-dockercfg-dx885\""
Sep 10 15:27:42.921714 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.921573    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-2e2b69\""
Sep 10 15:27:42.928895 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:42.928868    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"]
Sep 10 15:27:43.035443 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.035344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103b736c-7219-4989-b108-3a96e29bb1cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:43.035624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.035536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkzg2\" (UniqueName: \"kubernetes.io/projected/103b736c-7219-4989-b108-3a96e29bb1cd-kube-api-access-wkzg2\") pod \"payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:43.136193 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.136150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkzg2\" (UniqueName: \"kubernetes.io/projected/103b736c-7219-4989-b108-3a96e29bb1cd-kube-api-access-wkzg2\") pod \"payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:43.136363 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.136220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103b736c-7219-4989-b108-3a96e29bb1cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:43.137158 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.137133    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103b736c-7219-4989-b108-3a96e29bb1cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:43.145049 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.145024    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkzg2\" (UniqueName: \"kubernetes.io/projected/103b736c-7219-4989-b108-3a96e29bb1cd-kube-api-access-wkzg2\") pod \"payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:43.229625 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.229570    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:43.369223 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.369189    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf" event={"ID":"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8","Type":"ContainerStarted","Data":"488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff"}
Sep 10 15:27:43.369482 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.369292    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:43.370919 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.370895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:43.387396 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.386639    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf" podStartSLOduration=2.3866168 podStartE2EDuration="2.3866168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:43.385936543 +0000 UTC m=+703.921892131" watchObservedRunningTime="2026-09-10 15:27:43.3866168 +0000 UTC m=+703.922572365"
Sep 10 15:27:43.501520 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:43.501481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"]
Sep 10 15:27:43.504541 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:27:43.504509    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103b736c_7219_4989_b108_3a96e29bb1cd.slice/crio-39156d40759a67fc26d9b1e1b8f20eece334e50184d199ac2098bdad8d63651e WatchSource:0}: Error finding container 39156d40759a67fc26d9b1e1b8f20eece334e50184d199ac2098bdad8d63651e: Status 404 returned error can't find the container with id 39156d40759a67fc26d9b1e1b8f20eece334e50184d199ac2098bdad8d63651e
Sep 10 15:27:44.179224 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:44.179189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:44.376490 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:44.376438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb" event={"ID":"103b736c-7219-4989-b108-3a96e29bb1cd","Type":"ContainerStarted","Data":"23e3da3a6d48bc1b637485ce1a45318e99d5be07e8fc1c024c617f7749f57790"}
Sep 10 15:27:44.376693 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:44.376662    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:44.376751 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:44.376712    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb" event={"ID":"103b736c-7219-4989-b108-3a96e29bb1cd","Type":"ContainerStarted","Data":"39156d40759a67fc26d9b1e1b8f20eece334e50184d199ac2098bdad8d63651e"}
Sep 10 15:27:44.398125 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:44.398068    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb" podStartSLOduration=2.3980511890000002 podStartE2EDuration="2.398051189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:27:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:44.396290324 +0000 UTC m=+704.932245899" watchObservedRunningTime="2026-09-10 15:27:44.398051189 +0000 UTC m=+704.934006751"
Sep 10 15:27:45.440160 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:45.440120    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:27:46.334619 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:46.334583    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:27:48.760482 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:48.760430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:48.909104 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:48.909070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:49.007124 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:49.007082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:49.101965 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:49.101932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:51.784109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:51.784075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:53.475300 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:53.475216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:53.570828 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:53.570798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:54.378609 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:54.378574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:27:54.390805 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:54.390765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:55.383655 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:55.383620    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:27:56.777267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:56.777233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:57.083930 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:57.083899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:57.464645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:57.464556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:57.866460 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:27:57.866425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:01.110698 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:01.110665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:01.213167 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:01.213129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:01.744636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:01.744603    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:28:02.032507 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:02.032399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:03.003094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.003061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:03.084441 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.084407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:03.636519 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.636484    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"]
Sep 10 15:28:03.640087 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.640070    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.642991 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.642966    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-50fe56-dockercfg-z8j4c\""
Sep 10 15:28:03.643109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.643025    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-50fe56\""
Sep 10 15:28:03.654314 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.654287    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"]
Sep 10 15:28:03.668393 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.668361    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"]
Sep 10 15:28:03.672621 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.672604    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:03.686151 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.686124    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"]
Sep 10 15:28:03.748999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.748955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eb3962f-05b7-476b-99f5-10eb75537df6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.748999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.748998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9wxf\" (UniqueName: \"kubernetes.io/projected/5eb3962f-05b7-476b-99f5-10eb75537df6-kube-api-access-t9wxf\") pod \"payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.767712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.767672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:03.839183 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.839136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:03.850380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.850343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eb3962f-05b7-476b-99f5-10eb75537df6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.850597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.850391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wxf\" (UniqueName: \"kubernetes.io/projected/5eb3962f-05b7-476b-99f5-10eb75537df6-kube-api-access-t9wxf\") pod \"payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.850597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.850477    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:03.850597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.850507    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgq68\" (UniqueName: \"kubernetes.io/projected/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-kube-api-access-vgq68\") pod \"payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:03.851099 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.851080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eb3962f-05b7-476b-99f5-10eb75537df6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.861843 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.861821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9wxf\" (UniqueName: \"kubernetes.io/projected/5eb3962f-05b7-476b-99f5-10eb75537df6-kube-api-access-t9wxf\") pod \"payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.951315 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.951144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:03.951663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.951588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:03.951663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.951654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgq68\" (UniqueName: \"kubernetes.io/projected/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-kube-api-access-vgq68\") pod \"payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:03.953086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.953062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:03.968104 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.968078    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgq68\" (UniqueName: \"kubernetes.io/projected/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-kube-api-access-vgq68\") pod \"payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:03.984004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:03.983969    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:04.044984 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.044717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:04.232474 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.232435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"]
Sep 10 15:28:04.233936 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:28:04.233914    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb3962f_05b7_476b_99f5_10eb75537df6.slice/crio-14fa9602e0929e10195c618e81ae02882eb1e194a4d7a2cbc0ea0ad827d8d08f WatchSource:0}: Error finding container 14fa9602e0929e10195c618e81ae02882eb1e194a4d7a2cbc0ea0ad827d8d08f: Status 404 returned error can't find the container with id 14fa9602e0929e10195c618e81ae02882eb1e194a4d7a2cbc0ea0ad827d8d08f
Sep 10 15:28:04.275022 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.274998    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"]
Sep 10 15:28:04.276777 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:28:04.276742    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9225f8a5_a47a_4757_adb1_8459a0d0b0c7.slice/crio-721e677d9692950dbcd52cbae27dc94892b06aaf5f5ac08d2f9217017922b614 WatchSource:0}: Error finding container 721e677d9692950dbcd52cbae27dc94892b06aaf5f5ac08d2f9217017922b614: Status 404 returned error can't find the container with id 721e677d9692950dbcd52cbae27dc94892b06aaf5f5ac08d2f9217017922b614
Sep 10 15:28:04.484675 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.484579    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm" event={"ID":"9225f8a5-a47a-4757-adb1-8459a0d0b0c7","Type":"ContainerStarted","Data":"a5b5bbb5b29b117887d67ea303689da0224fe81f7cbcd1d8363e53b3e23f40f5"}
Sep 10 15:28:04.484675 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.484649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm" event={"ID":"9225f8a5-a47a-4757-adb1-8459a0d0b0c7","Type":"ContainerStarted","Data":"721e677d9692950dbcd52cbae27dc94892b06aaf5f5ac08d2f9217017922b614"}
Sep 10 15:28:04.484891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.484694    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:04.486385 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.486357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7" event={"ID":"5eb3962f-05b7-476b-99f5-10eb75537df6","Type":"ContainerStarted","Data":"49c4ec8677b1efd2791a348f7426f9d5ab18897eed3f75c053b920cdbf7632a4"}
Sep 10 15:28:04.486588 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.486406    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7" event={"ID":"5eb3962f-05b7-476b-99f5-10eb75537df6","Type":"ContainerStarted","Data":"14fa9602e0929e10195c618e81ae02882eb1e194a4d7a2cbc0ea0ad827d8d08f"}
Sep 10 15:28:04.486588 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.486495    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:04.509166 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.509117    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm" podStartSLOduration=1.50910324 podStartE2EDuration="1.50910324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:04.506646156 +0000 UTC m=+725.042601740" watchObservedRunningTime="2026-09-10 15:28:04.50910324 +0000 UTC m=+725.045059201"
Sep 10 15:28:04.526562 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:04.526508    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7" podStartSLOduration=1.5264903890000001 podStartE2EDuration="1.526490389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:04.52565277 +0000 UTC m=+725.061608333" watchObservedRunningTime="2026-09-10 15:28:04.526490389 +0000 UTC m=+725.062445951"
Sep 10 15:28:05.047045 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:05.047008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:05.956248 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:05.956212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:06.919998 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:06.919962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:07.011362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:07.011329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:07.210596 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:07.210502    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:07.774650 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:07.774615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:09.967939 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:09.967895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:11.591941 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.591904    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"]
Sep 10 15:28:11.596793 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.596776    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.600069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.600046    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-4d992b9c-kserve-self-signed-certs\""
Sep 10 15:28:11.605944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.605922    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"]
Sep 10 15:28:11.625702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.625668    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-home\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.625702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.625702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tls-certs\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.625908 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.625834    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-model-cache\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.625908 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.625874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-dshm\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.626020 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.625925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tmp-dir\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.626020 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.625957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvnh\" (UniqueName: \"kubernetes.io/projected/ea22da64-1220-47b1-94ad-3fd33ba00cdc-kube-api-access-2qvnh\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.726546 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-model-cache\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.726546 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-dshm\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.726794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tmp-dir\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.726794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvnh\" (UniqueName: \"kubernetes.io/projected/ea22da64-1220-47b1-94ad-3fd33ba00cdc-kube-api-access-2qvnh\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.726794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726673    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-home\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.726794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tls-certs\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.727006 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-model-cache\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.727006 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.726962    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tmp-dir\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.727117 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.727034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-home\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.728771 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.728749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-dshm\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.729078 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.729061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tls-certs\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.734772 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.734742    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:11.736164 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.736143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvnh\" (UniqueName: \"kubernetes.io/projected/ea22da64-1220-47b1-94ad-3fd33ba00cdc-kube-api-access-2qvnh\") pod \"e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") " pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:11.907018 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:11.906921    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:12.184553 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:12.184527    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"]
Sep 10 15:28:12.186536 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:28:12.186509    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea22da64_1220_47b1_94ad_3fd33ba00cdc.slice/crio-5fa3fb4d3880170da605ff16f3bb85cd5c95c752c8ad34e8d1a0f8e148a3f4f8 WatchSource:0}: Error finding container 5fa3fb4d3880170da605ff16f3bb85cd5c95c752c8ad34e8d1a0f8e148a3f4f8: Status 404 returned error can't find the container with id 5fa3fb4d3880170da605ff16f3bb85cd5c95c752c8ad34e8d1a0f8e148a3f4f8
Sep 10 15:28:12.529321 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:12.529212    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58" event={"ID":"ea22da64-1220-47b1-94ad-3fd33ba00cdc","Type":"ContainerStarted","Data":"1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af"}
Sep 10 15:28:12.529321 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:12.529259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58" event={"ID":"ea22da64-1220-47b1-94ad-3fd33ba00cdc","Type":"ContainerStarted","Data":"5fa3fb4d3880170da605ff16f3bb85cd5c95c752c8ad34e8d1a0f8e148a3f4f8"}
Sep 10 15:28:12.551295 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:12.551242    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58" podStartSLOduration=1.551226342 podStartE2EDuration="1.551226342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:12.549182627 +0000 UTC m=+733.085138213" watchObservedRunningTime="2026-09-10 15:28:12.551226342 +0000 UTC m=+733.087181906"
Sep 10 15:28:13.331319 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:13.331281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:13.993926 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:13.993892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:14.059831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:14.059769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:15.494660 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:15.494621    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:28:15.495081 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:15.494720    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:28:18.150467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:18.150424    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:28:20.742908 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:20.742873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.397015 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.396969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.497595 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.497561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.629242 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.629210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.679478 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.679375    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"]
Sep 10 15:28:21.679670 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.679642    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf" podUID="e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" containerName="payload-processing" containerID="cri-o://488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff" gracePeriod=30
Sep 10 15:28:21.717008 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.716945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.800508 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.800476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.890878 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.890847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.907998 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.907964    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:21.908185 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.908014    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:21.924351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.924270    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:21.940958 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.940903    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:28:21.967215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:21.967187    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.019411 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.019377    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z52g\" (UniqueName: \"kubernetes.io/projected/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-kube-api-access-4z52g\") pod \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") "
Sep 10 15:28:22.020074 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.019997    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-plugins-config-volume\") pod \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\" (UID: \"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8\") "
Sep 10 15:28:22.020499 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.020214    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" (UID: "e4ab0cec-1730-4eb2-aa15-978b80bcaaa8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:28:22.021001 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.020976    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:28:22.022850 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.022813    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-kube-api-access-4z52g" (OuterVolumeSpecName: "kube-api-access-4z52g") pod "e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" (UID: "e4ab0cec-1730-4eb2-aa15-978b80bcaaa8"). InnerVolumeSpecName "kube-api-access-4z52g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:28:22.035672 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.035644    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.122487 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.122445    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z52g\" (UniqueName: \"kubernetes.io/projected/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8-kube-api-access-4z52g\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:28:22.509797 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.509761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.586078 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.586041    2562 generic.go:356] "Generic (PLEG): container finished" podID="e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" containerID="488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff" exitCode=0
Sep 10 15:28:22.586251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.586086    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"
Sep 10 15:28:22.586251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.586123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf" event={"ID":"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8","Type":"ContainerDied","Data":"488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff"}
Sep 10 15:28:22.586251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.586171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf" event={"ID":"e4ab0cec-1730-4eb2-aa15-978b80bcaaa8","Type":"ContainerDied","Data":"5525bcfa5a018342af06c6f43126164f0d65f9fc478b1c0e71dc079823529698"}
Sep 10 15:28:22.586251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.586193    2562 scope.go:117] "RemoveContainer" containerID="488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff"
Sep 10 15:28:22.595230 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.595215    2562 scope.go:117] "RemoveContainer" containerID="488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff"
Sep 10 15:28:22.595606 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:28:22.595579    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff\": container with ID starting with 488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff not found: ID does not exist" containerID="488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff"
Sep 10 15:28:22.595681 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.595614    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff"} err="failed to get container status \"488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff\": rpc error: code = NotFound desc = could not find container \"488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff\": container with ID starting with 488a0a7b6c1685c791726f61c94db710ff1aa60abff20b6708be3324bfccb8ff not found: ID does not exist"
Sep 10 15:28:22.599235 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.599215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.600827 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.600803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:28:22.610969 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.610948    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"]
Sep 10 15:28:22.614972 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.614947    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b74e71b-684b6dd54c-q8xsf"]
Sep 10 15:28:22.799913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:22.799796    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:23.027707 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:23.027655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:23.382647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:23.382573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:23.956222 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:23.956187    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" path="/var/lib/kubelet/pods/e4ab0cec-1730-4eb2-aa15-978b80bcaaa8/volumes"
Sep 10 15:28:23.998702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:23.998653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.042636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.042594    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.122037 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.121994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.209144 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.209112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.274795 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.274759    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.547072 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.547042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.768078 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.768036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.859613 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.859573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.868228 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.868201    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"]
Sep 10 15:28:26.868719 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.868704    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" containerName="payload-processing"
Sep 10 15:28:26.868766 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.868721    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" containerName="payload-processing"
Sep 10 15:28:26.868800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.868791    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ab0cec-1730-4eb2-aa15-978b80bcaaa8" containerName="payload-processing"
Sep 10 15:28:26.871904 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.871886    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:26.874257 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.874240    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-8d054c9d\""
Sep 10 15:28:26.884179 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.884159    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"]
Sep 10 15:28:26.973354 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.973313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd689\" (UniqueName: \"kubernetes.io/projected/aded1ce0-217b-4307-8410-d683917ab95b-kube-api-access-bd689\") pod \"payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:26.973548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:26.973389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aded1ce0-217b-4307-8410-d683917ab95b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:27.074097 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.074065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd689\" (UniqueName: \"kubernetes.io/projected/aded1ce0-217b-4307-8410-d683917ab95b-kube-api-access-bd689\") pod \"payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:27.074561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.074154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aded1ce0-217b-4307-8410-d683917ab95b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:27.074940 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.074916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aded1ce0-217b-4307-8410-d683917ab95b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:27.082549 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.082523    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd689\" (UniqueName: \"kubernetes.io/projected/aded1ce0-217b-4307-8410-d683917ab95b-kube-api-access-bd689\") pod \"payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:27.182922 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.182841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:27.461502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.461474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"]
Sep 10 15:28:27.463332 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:28:27.463307    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaded1ce0_217b_4307_8410_d683917ab95b.slice/crio-df4915e5a912751f009f3fff64fa96c9ddf915383cd92ff4a2073385594f5cc4 WatchSource:0}: Error finding container df4915e5a912751f009f3fff64fa96c9ddf915383cd92ff4a2073385594f5cc4: Status 404 returned error can't find the container with id df4915e5a912751f009f3fff64fa96c9ddf915383cd92ff4a2073385594f5cc4
Sep 10 15:28:27.620320 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.620284    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6" event={"ID":"aded1ce0-217b-4307-8410-d683917ab95b","Type":"ContainerStarted","Data":"585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f"}
Sep 10 15:28:27.620504 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.620336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6" event={"ID":"aded1ce0-217b-4307-8410-d683917ab95b","Type":"ContainerStarted","Data":"df4915e5a912751f009f3fff64fa96c9ddf915383cd92ff4a2073385594f5cc4"}
Sep 10 15:28:27.620504 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.620387    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:27.639828 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:27.639782    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6" podStartSLOduration=1.639767497 podStartE2EDuration="1.639767497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:27.638325061 +0000 UTC m=+748.174280647" watchObservedRunningTime="2026-09-10 15:28:27.639767497 +0000 UTC m=+748.175723093"
Sep 10 15:28:35.437390 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:35.437351    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:28:38.627149 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:38.627119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:42.473876 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:42.473842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:49.241193 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:49.241155    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:28:57.812218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:57.812175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:57.855080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:57.855041    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"]
Sep 10 15:28:57.855327 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:57.855279    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6" podUID="aded1ce0-217b-4307-8410-d683917ab95b" containerName="payload-pre-processing" containerID="cri-o://585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f" gracePeriod=30
Sep 10 15:28:57.881020 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:57.880992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:57.958496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:57.958464    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.052085 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.052036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.111010 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.110982    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:58.117180 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.117138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.197043 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.197009    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd689\" (UniqueName: \"kubernetes.io/projected/aded1ce0-217b-4307-8410-d683917ab95b-kube-api-access-bd689\") pod \"aded1ce0-217b-4307-8410-d683917ab95b\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") "
Sep 10 15:28:58.197043 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.197045    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aded1ce0-217b-4307-8410-d683917ab95b-plugins-config-volume\") pod \"aded1ce0-217b-4307-8410-d683917ab95b\" (UID: \"aded1ce0-217b-4307-8410-d683917ab95b\") "
Sep 10 15:28:58.197475 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.197425    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aded1ce0-217b-4307-8410-d683917ab95b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aded1ce0-217b-4307-8410-d683917ab95b" (UID: "aded1ce0-217b-4307-8410-d683917ab95b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:28:58.199095 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.199076    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aded1ce0-217b-4307-8410-d683917ab95b-kube-api-access-bd689" (OuterVolumeSpecName: "kube-api-access-bd689") pod "aded1ce0-217b-4307-8410-d683917ab95b" (UID: "aded1ce0-217b-4307-8410-d683917ab95b"). InnerVolumeSpecName "kube-api-access-bd689". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:28:58.202891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.202855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.298536 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.298511    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd689\" (UniqueName: \"kubernetes.io/projected/aded1ce0-217b-4307-8410-d683917ab95b-kube-api-access-bd689\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:28:58.298536 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.298535    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aded1ce0-217b-4307-8410-d683917ab95b-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:28:58.624346 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.624285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.785609 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.785578    2562 generic.go:356] "Generic (PLEG): container finished" podID="aded1ce0-217b-4307-8410-d683917ab95b" containerID="585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f" exitCode=0
Sep 10 15:28:58.785800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.785620    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"
Sep 10 15:28:58.785800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.785663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6" event={"ID":"aded1ce0-217b-4307-8410-d683917ab95b","Type":"ContainerDied","Data":"585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f"}
Sep 10 15:28:58.785800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.785721    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6" event={"ID":"aded1ce0-217b-4307-8410-d683917ab95b","Type":"ContainerDied","Data":"df4915e5a912751f009f3fff64fa96c9ddf915383cd92ff4a2073385594f5cc4"}
Sep 10 15:28:58.785800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.785750    2562 scope.go:117] "RemoveContainer" containerID="585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f"
Sep 10 15:28:58.794967 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.794951    2562 scope.go:117] "RemoveContainer" containerID="585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f"
Sep 10 15:28:58.795242 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:28:58.795224    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f\": container with ID starting with 585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f not found: ID does not exist" containerID="585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f"
Sep 10 15:28:58.795287 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.795251    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f"} err="failed to get container status \"585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f\": rpc error: code = NotFound desc = could not find container \"585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f\": container with ID starting with 585e6e982384163eeb7fb501f87a868e137c9760a43cb1cd9354f322cddc895f not found: ID does not exist"
Sep 10 15:28:58.807215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.807182    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"]
Sep 10 15:28:58.809507 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.809489    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8d054c9d-54764bcb7d-c6vz6"]
Sep 10 15:28:58.957419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:58.957332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:59.088270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:59.088235    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:59.955582 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:28:59.955557    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aded1ce0-217b-4307-8410-d683917ab95b" path="/var/lib/kubelet/pods/aded1ce0-217b-4307-8410-d683917ab95b/volumes"
Sep 10 15:29:00.954138 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:00.954104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:01.048621 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:01.048592    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:02.277084 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.277045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:02.377252 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.377221    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:02.849814 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.849783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:02.953667 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.953631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:02.962630 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.962598    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"]
Sep 10 15:29:02.963048 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.963034    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aded1ce0-217b-4307-8410-d683917ab95b" containerName="payload-pre-processing"
Sep 10 15:29:02.963092 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.963049    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aded1ce0-217b-4307-8410-d683917ab95b" containerName="payload-pre-processing"
Sep 10 15:29:02.963131 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.963117    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aded1ce0-217b-4307-8410-d683917ab95b" containerName="payload-pre-processing"
Sep 10 15:29:02.967486 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.967444    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:02.969982 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.969964    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9fd15233\""
Sep 10 15:29:02.975887 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:02.975865    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"]
Sep 10 15:29:03.036393 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.036349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:03.048848 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.048823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.048974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.048876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdx26\" (UniqueName: \"kubernetes.io/projected/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-kube-api-access-kdx26\") pod \"payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.150318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.150236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdx26\" (UniqueName: \"kubernetes.io/projected/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-kube-api-access-kdx26\") pod \"payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.150488 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.150343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.151024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.151005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.162395 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.162374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdx26\" (UniqueName: \"kubernetes.io/projected/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-kube-api-access-kdx26\") pod \"payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.279552 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.279514    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.555502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.555471    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"]
Sep 10 15:29:03.556822 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:29:03.556796    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ba2197_a9bc_4c2a_9f23_72ab4d4b65f1.slice/crio-c52f1aa25adc1f231c843defa47020c7062328dc5302974bb473937acea2d2cf WatchSource:0}: Error finding container c52f1aa25adc1f231c843defa47020c7062328dc5302974bb473937acea2d2cf: Status 404 returned error can't find the container with id c52f1aa25adc1f231c843defa47020c7062328dc5302974bb473937acea2d2cf
Sep 10 15:29:03.813261 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.813229    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk" event={"ID":"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1","Type":"ContainerStarted","Data":"eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf"}
Sep 10 15:29:03.813400 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.813282    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk" event={"ID":"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1","Type":"ContainerStarted","Data":"c52f1aa25adc1f231c843defa47020c7062328dc5302974bb473937acea2d2cf"}
Sep 10 15:29:03.813400 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.813311    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:03.831523 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:03.831473    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk" podStartSLOduration=1.831441606 podStartE2EDuration="1.831441606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:29:03.830753936 +0000 UTC m=+784.366709512" watchObservedRunningTime="2026-09-10 15:29:03.831441606 +0000 UTC m=+784.367397169"
Sep 10 15:29:07.329134 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:07.329098    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:29:10.771348 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:10.771316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:14.823647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:14.823618    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:17.860316 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:17.860278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:23.319174 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:23.319069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:25.329619 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:25.329577    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:29:27.987847 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:27.987810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:28.031278 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.031242    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"]
Sep 10 15:29:28.035217 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.035188    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.038104 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.038070    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-4d992b9c-kserve-self-signed-certs\""
Sep 10 15:29:28.047927 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.047898    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"]
Sep 10 15:29:28.064767 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.064733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:28.107489 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.107439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tmp-dir\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.107658 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.107548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-model-cache\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.107658 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.107597    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tls-certs\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.107751 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.107694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q687w\" (UniqueName: \"kubernetes.io/projected/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-kube-api-access-q687w\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.107793 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.107767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-home\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.107854 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.107839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-dshm\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.208818 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.208778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q687w\" (UniqueName: \"kubernetes.io/projected/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-kube-api-access-q687w\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209035 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.208842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-home\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209035 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.208894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-dshm\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209035 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.208924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tmp-dir\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209035 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.208993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-model-cache\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209035 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.209022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tls-certs\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209330 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.209304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-home\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.209316    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tmp-dir\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.209419 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.209400    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-model-cache\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.211370 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.211344    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-dshm\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.211494 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.211477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tls-certs\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.217928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.217910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q687w\" (UniqueName: \"kubernetes.io/projected/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-kube-api-access-q687w\") pod \"e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") " pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.345590 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.345555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:28.629107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.629080    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"]
Sep 10 15:29:28.631232 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:29:28.631204    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb6e5eca_a37a_4b63_bad1_92a4fd89dd83.slice/crio-d9ab5902dab4dc24b641a27fc4a2968c60a2b51158e1a9d8c946126ceec05154 WatchSource:0}: Error finding container d9ab5902dab4dc24b641a27fc4a2968c60a2b51158e1a9d8c946126ceec05154: Status 404 returned error can't find the container with id d9ab5902dab4dc24b641a27fc4a2968c60a2b51158e1a9d8c946126ceec05154
Sep 10 15:29:28.721648 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.721618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:28.777374 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.777336    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"]
Sep 10 15:29:28.777643 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.777612    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb" podUID="103b736c-7219-4989-b108-3a96e29bb1cd" containerName="payload-pre-processing" containerID="cri-o://23e3da3a6d48bc1b637485ce1a45318e99d5be07e8fc1c024c617f7749f57790" gracePeriod=30
Sep 10 15:29:28.801708 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.801681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:28.866251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.866220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:28.946760 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.946722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs" event={"ID":"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83","Type":"ContainerStarted","Data":"cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1"}
Sep 10 15:29:28.946936 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.946769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs" event={"ID":"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83","Type":"ContainerStarted","Data":"d9ab5902dab4dc24b641a27fc4a2968c60a2b51158e1a9d8c946126ceec05154"}
Sep 10 15:29:28.948444 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.948419    2562 generic.go:356] "Generic (PLEG): container finished" podID="103b736c-7219-4989-b108-3a96e29bb1cd" containerID="23e3da3a6d48bc1b637485ce1a45318e99d5be07e8fc1c024c617f7749f57790" exitCode=0
Sep 10 15:29:28.948590 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.948489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb" event={"ID":"103b736c-7219-4989-b108-3a96e29bb1cd","Type":"ContainerDied","Data":"23e3da3a6d48bc1b637485ce1a45318e99d5be07e8fc1c024c617f7749f57790"}
Sep 10 15:29:28.951323 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.951294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:28.967628 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:28.967578    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs" podStartSLOduration=0.967560698 podStartE2EDuration="967.560698ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:29:28.965147051 +0000 UTC m=+809.501102615" watchObservedRunningTime="2026-09-10 15:29:28.967560698 +0000 UTC m=+809.503516261"
Sep 10 15:29:29.019492 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.019463    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:29:29.021669 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.021645    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:29.120061 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.120022    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkzg2\" (UniqueName: \"kubernetes.io/projected/103b736c-7219-4989-b108-3a96e29bb1cd-kube-api-access-wkzg2\") pod \"103b736c-7219-4989-b108-3a96e29bb1cd\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") "
Sep 10 15:29:29.120249 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.120221    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103b736c-7219-4989-b108-3a96e29bb1cd-plugins-config-volume\") pod \"103b736c-7219-4989-b108-3a96e29bb1cd\" (UID: \"103b736c-7219-4989-b108-3a96e29bb1cd\") "
Sep 10 15:29:29.120599 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.120574    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/103b736c-7219-4989-b108-3a96e29bb1cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "103b736c-7219-4989-b108-3a96e29bb1cd" (UID: "103b736c-7219-4989-b108-3a96e29bb1cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:29.122085 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.122064    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/103b736c-7219-4989-b108-3a96e29bb1cd-kube-api-access-wkzg2" (OuterVolumeSpecName: "kube-api-access-wkzg2") pod "103b736c-7219-4989-b108-3a96e29bb1cd" (UID: "103b736c-7219-4989-b108-3a96e29bb1cd"). InnerVolumeSpecName "kube-api-access-wkzg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:29:29.221896 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.221865    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103b736c-7219-4989-b108-3a96e29bb1cd-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:29:29.221896 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.221898    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkzg2\" (UniqueName: \"kubernetes.io/projected/103b736c-7219-4989-b108-3a96e29bb1cd-kube-api-access-wkzg2\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:29:29.313241 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.313202    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:29.955412 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.955385    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"
Sep 10 15:29:29.956686 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.956661    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb" event={"ID":"103b736c-7219-4989-b108-3a96e29bb1cd","Type":"ContainerDied","Data":"39156d40759a67fc26d9b1e1b8f20eece334e50184d199ac2098bdad8d63651e"}
Sep 10 15:29:29.956831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.956707    2562 scope.go:117] "RemoveContainer" containerID="23e3da3a6d48bc1b637485ce1a45318e99d5be07e8fc1c024c617f7749f57790"
Sep 10 15:29:29.980830 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.980802    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"]
Sep 10 15:29:29.992104 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:29.992077    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2e2b69-85c57cfc8b-rrxzb"]
Sep 10 15:29:31.955761 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:31.955730    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="103b736c-7219-4989-b108-3a96e29bb1cd" path="/var/lib/kubelet/pods/103b736c-7219-4989-b108-3a96e29bb1cd/volumes"
Sep 10 15:29:32.376057 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:32.376027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:32.971718 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:32.971687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:33.014236 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.014202    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"]
Sep 10 15:29:33.014498 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.014427    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk" podUID="d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" containerName="payload-pre-processing" containerID="cri-o://eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf" gracePeriod=30
Sep 10 15:29:33.048379 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.048346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:33.149305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.149273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:33.212883 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.212850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:33.273017 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.272991    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:33.364988 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.364952    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdx26\" (UniqueName: \"kubernetes.io/projected/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-kube-api-access-kdx26\") pod \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") "
Sep 10 15:29:33.364988 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.364992    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-plugins-config-volume\") pod \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\" (UID: \"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1\") "
Sep 10 15:29:33.365443 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.365408    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" (UID: "d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:33.367143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.367117    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-kube-api-access-kdx26" (OuterVolumeSpecName: "kube-api-access-kdx26") pod "d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" (UID: "d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1"). InnerVolumeSpecName "kube-api-access-kdx26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:29:33.465773 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.465729    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdx26\" (UniqueName: \"kubernetes.io/projected/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-kube-api-access-kdx26\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:29:33.465773 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.465769    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:29:33.977616 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.977579    2562 generic.go:356] "Generic (PLEG): container finished" podID="d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" containerID="eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf" exitCode=0
Sep 10 15:29:33.978026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.977623    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"
Sep 10 15:29:33.978026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.977670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk" event={"ID":"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1","Type":"ContainerDied","Data":"eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf"}
Sep 10 15:29:33.978026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.977733    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk" event={"ID":"d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1","Type":"ContainerDied","Data":"c52f1aa25adc1f231c843defa47020c7062328dc5302974bb473937acea2d2cf"}
Sep 10 15:29:33.978026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.977768    2562 scope.go:117] "RemoveContainer" containerID="eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf"
Sep 10 15:29:33.986918 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.986897    2562 scope.go:117] "RemoveContainer" containerID="eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf"
Sep 10 15:29:33.987182 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:29:33.987162    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf\": container with ID starting with eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf not found: ID does not exist" containerID="eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf"
Sep 10 15:29:33.987243 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.987193    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf"} err="failed to get container status \"eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf\": rpc error: code = NotFound desc = could not find container \"eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf\": container with ID starting with eca88ba350fa59555faaf6646ca65d1e37bc2f1a974bf217c414e4a537aed0cf not found: ID does not exist"
Sep 10 15:29:33.997427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:33.997396    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"]
Sep 10 15:29:34.000609 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:34.000584    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9fd15233-6b8c9f49fd-q85vk"]
Sep 10 15:29:34.298251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:34.298167    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:34.370568 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:34.370529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:34.716619 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:34.716587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:35.017282 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:35.017192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:35.514568 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:35.514532    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:35.955263 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:35.955225    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" path="/var/lib/kubelet/pods/d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1/volumes"
Sep 10 15:29:37.224615 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:37.224580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:37.317690 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:37.317648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:38.346379 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:38.346347    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:38.346823 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:38.346494    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:38.359385 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:38.359352    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:38.468146 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:38.468112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:38.568522 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:38.568486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:38.668411 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:38.668332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:38.973346 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:38.973265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:39.017241 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.017211    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:29:39.068781 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.068746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:39.143934 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.143905    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"]
Sep 10 15:29:39.144355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.144342    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103b736c-7219-4989-b108-3a96e29bb1cd" containerName="payload-pre-processing"
Sep 10 15:29:39.144404 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.144358    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="103b736c-7219-4989-b108-3a96e29bb1cd" containerName="payload-pre-processing"
Sep 10 15:29:39.144404 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.144382    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" containerName="payload-pre-processing"
Sep 10 15:29:39.144404 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.144388    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" containerName="payload-pre-processing"
Sep 10 15:29:39.144525 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.144473    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="103b736c-7219-4989-b108-3a96e29bb1cd" containerName="payload-pre-processing"
Sep 10 15:29:39.144525 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.144484    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5ba2197-a9bc-4c2a-9f23-72ab4d4b65f1" containerName="payload-pre-processing"
Sep 10 15:29:39.148880 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.148863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.151328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.151306    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-f5d25c39\""
Sep 10 15:29:39.151447 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.151399    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-f5d25c39-dockercfg-mg6j5\""
Sep 10 15:29:39.159050 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.159029    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"]
Sep 10 15:29:39.235071 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.234976    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.235071 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.235033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zjn2\" (UniqueName: \"kubernetes.io/projected/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-kube-api-access-2zjn2\") pod \"payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.335771 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.335726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.335984 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.335812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zjn2\" (UniqueName: \"kubernetes.io/projected/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-kube-api-access-2zjn2\") pod \"payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.336432 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.336412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.344184 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.344160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zjn2\" (UniqueName: \"kubernetes.io/projected/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-kube-api-access-2zjn2\") pod \"payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.460895 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.460854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:39.735265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:39.735233    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"]
Sep 10 15:29:39.738096 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:29:39.738067    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6cad77_2d22_45a3_8f89_8e2d94ebeb8a.slice/crio-5167e551338a0af701a0393df11501b6597917dd6e7662f589d5bf54100b0bdb WatchSource:0}: Error finding container 5167e551338a0af701a0393df11501b6597917dd6e7662f589d5bf54100b0bdb: Status 404 returned error can't find the container with id 5167e551338a0af701a0393df11501b6597917dd6e7662f589d5bf54100b0bdb
Sep 10 15:29:40.012370 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:40.012328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh" event={"ID":"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a","Type":"ContainerStarted","Data":"fad2a37ce3afd53aa08eb4eb701b5799deda3ee1a6ddc933975139664273607a"}
Sep 10 15:29:40.012595 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:40.012389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh" event={"ID":"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a","Type":"ContainerStarted","Data":"5167e551338a0af701a0393df11501b6597917dd6e7662f589d5bf54100b0bdb"}
Sep 10 15:29:40.012595 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:40.012496    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:40.032396 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:40.032273    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh" podStartSLOduration=1.032255409 podStartE2EDuration="1.032255409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:29:40.029566818 +0000 UTC m=+820.565522401" watchObservedRunningTime="2026-09-10 15:29:40.032255409 +0000 UTC m=+820.568210971"
Sep 10 15:29:42.033768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:42.033733    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:29:51.020395 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:51.020352    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:29:56.429811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:29:56.429774    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:30:00.148011 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.147961    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"]
Sep 10 15:30:00.152783 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.152759    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"]
Sep 10 15:30:00.152941 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.152901    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"
Sep 10 15:30:00.155561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.155538    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:30:00.156117 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.156100    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"
Sep 10 15:30:00.163334 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.163310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"
Sep 10 15:30:00.168360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.168330    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"]
Sep 10 15:30:00.171874 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.171847    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"]
Sep 10 15:30:00.172985 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.172965    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"
Sep 10 15:30:00.453165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.453132    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"]
Sep 10 15:30:00.454708 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:30:00.454680    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ade48de_2f8e_451c_9ea9_a88fd53ff756.slice/crio-d3adabdeec4a8576347b779e97f445160295f6eab7fa14484a5ba6180edcd9a6 WatchSource:0}: Error finding container d3adabdeec4a8576347b779e97f445160295f6eab7fa14484a5ba6180edcd9a6: Status 404 returned error can't find the container with id d3adabdeec4a8576347b779e97f445160295f6eab7fa14484a5ba6180edcd9a6
Sep 10 15:30:00.456945 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.456920    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:30:00.470394 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:00.470365    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"]
Sep 10 15:30:00.472574 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:30:00.472545    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce879c5_0d7e_48d6_905a_26625e4cb05c.slice/crio-e7bc5f99e7d6fb642551731eb8fcdb98b091ca49ceca4c4219a6174d4dfe702a WatchSource:0}: Error finding container e7bc5f99e7d6fb642551731eb8fcdb98b091ca49ceca4c4219a6174d4dfe702a: Status 404 returned error can't find the container with id e7bc5f99e7d6fb642551731eb8fcdb98b091ca49ceca4c4219a6174d4dfe702a
Sep 10 15:30:01.125068 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:01.125034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz" event={"ID":"3ce879c5-0d7e-48d6-905a-26625e4cb05c","Type":"ContainerStarted","Data":"e7bc5f99e7d6fb642551731eb8fcdb98b091ca49ceca4c4219a6174d4dfe702a"}
Sep 10 15:30:01.126244 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:01.126218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258" event={"ID":"0ade48de-2f8e-451c-9ea9-a88fd53ff756","Type":"ContainerStarted","Data":"d3adabdeec4a8576347b779e97f445160295f6eab7fa14484a5ba6180edcd9a6"}
Sep 10 15:30:01.938044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:01.937997    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:04.148425 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:04.148342    2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce879c5-0d7e-48d6-905a-26625e4cb05c" containerID="3bfd45e2f03fe00168bb764f1c4face71a286e009f0c3a7101a74bd69ca795cd" exitCode=0
Sep 10 15:30:04.148835 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:04.148430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz" event={"ID":"3ce879c5-0d7e-48d6-905a-26625e4cb05c","Type":"ContainerDied","Data":"3bfd45e2f03fe00168bb764f1c4face71a286e009f0c3a7101a74bd69ca795cd"}
Sep 10 15:30:04.150009 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:04.149986    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ade48de-2f8e-451c-9ea9-a88fd53ff756" containerID="02cb060cefe4308fe0f40d9670074d3c28a35335ea34a2d3fc0e7d7df9093ed9" exitCode=0
Sep 10 15:30:04.150132 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:04.150016    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258" event={"ID":"0ade48de-2f8e-451c-9ea9-a88fd53ff756","Type":"ContainerDied","Data":"02cb060cefe4308fe0f40d9670074d3c28a35335ea34a2d3fc0e7d7df9093ed9"}
Sep 10 15:30:04.224244 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:04.224211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:04.916744 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:04.916707    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"]
Sep 10 15:30:04.931868 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:04.931834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:05.289756 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:05.289731    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"
Sep 10 15:30:05.325190 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:05.325162    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"
Sep 10 15:30:05.616115 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:05.616071    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"]
Sep 10 15:30:05.641194 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:05.641163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:06.167472 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:06.167433    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"
Sep 10 15:30:06.167657 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:06.167476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz" event={"ID":"3ce879c5-0d7e-48d6-905a-26625e4cb05c","Type":"ContainerDied","Data":"e7bc5f99e7d6fb642551731eb8fcdb98b091ca49ceca4c4219a6174d4dfe702a"}
Sep 10 15:30:06.167657 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:06.167562    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7bc5f99e7d6fb642551731eb8fcdb98b091ca49ceca4c4219a6174d4dfe702a"
Sep 10 15:30:06.168903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:06.168878    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258" event={"ID":"0ade48de-2f8e-451c-9ea9-a88fd53ff756","Type":"ContainerDied","Data":"d3adabdeec4a8576347b779e97f445160295f6eab7fa14484a5ba6180edcd9a6"}
Sep 10 15:30:06.169021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:06.168913    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3adabdeec4a8576347b779e97f445160295f6eab7fa14484a5ba6180edcd9a6"
Sep 10 15:30:06.169021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:06.168888    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"
Sep 10 15:30:08.981497 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:08.981466    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:08.986232 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:08.986205    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"]
Sep 10 15:30:08.988146 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:08.988127    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-f5d25c39-29817570-sh7cz"]
Sep 10 15:30:09.021253 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.021221    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"]
Sep 10 15:30:09.021500 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.021471    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh" podUID="bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" containerName="payload-pre-processing" containerID="cri-o://fad2a37ce3afd53aa08eb4eb701b5799deda3ee1a6ddc933975139664273607a" gracePeriod=30
Sep 10 15:30:09.103631 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.103598    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:09.191010 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.190974    2562 generic.go:356] "Generic (PLEG): container finished" podID="bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" containerID="fad2a37ce3afd53aa08eb4eb701b5799deda3ee1a6ddc933975139664273607a" exitCode=0
Sep 10 15:30:09.191154 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.191043    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh" event={"ID":"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a","Type":"ContainerDied","Data":"fad2a37ce3afd53aa08eb4eb701b5799deda3ee1a6ddc933975139664273607a"}
Sep 10 15:30:09.195371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.195344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:09.256373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.256292    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:09.283044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.283020    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:30:09.339070 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.339025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:09.451882 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.451846    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zjn2\" (UniqueName: \"kubernetes.io/projected/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-kube-api-access-2zjn2\") pod \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") "
Sep 10 15:30:09.452076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.451904    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-plugins-config-volume\") pod \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\" (UID: \"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a\") "
Sep 10 15:30:09.452276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.452254    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" (UID: "bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:30:09.454019 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.453988    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-kube-api-access-2zjn2" (OuterVolumeSpecName: "kube-api-access-2zjn2") pod "bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" (UID: "bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a"). InnerVolumeSpecName "kube-api-access-2zjn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:30:09.553058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.552982    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zjn2\" (UniqueName: \"kubernetes.io/projected/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-kube-api-access-2zjn2\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:09.553058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.553014    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:09.956682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.956646    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ce879c5-0d7e-48d6-905a-26625e4cb05c" path="/var/lib/kubelet/pods/3ce879c5-0d7e-48d6-905a-26625e4cb05c/volumes"
Sep 10 15:30:09.968331 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:09.968298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:10.065383 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:10.065339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:10.132900 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:10.132866    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:10.197561 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:10.197529    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"
Sep 10 15:30:10.197733 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:10.197529    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh" event={"ID":"bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a","Type":"ContainerDied","Data":"5167e551338a0af701a0393df11501b6597917dd6e7662f589d5bf54100b0bdb"}
Sep 10 15:30:10.197733 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:10.197660    2562 scope.go:117] "RemoveContainer" containerID="fad2a37ce3afd53aa08eb4eb701b5799deda3ee1a6ddc933975139664273607a"
Sep 10 15:30:10.215110 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:10.215085    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"]
Sep 10 15:30:10.218774 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:10.218752    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f5d25c39-58f64c4d8b-z98xh"]
Sep 10 15:30:11.465528 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:11.465479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:11.955607 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:11.955574    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" path="/var/lib/kubelet/pods/bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a/volumes"
Sep 10 15:30:12.251145 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:12.251066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:12.509050 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:12.508956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:12.916635 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:12.916604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:13.176359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:13.176268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:13.247387 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:13.247353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:13.325869 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:13.325833    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:16.042409 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:16.042372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:16.138709 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:16.138675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:17.456279 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:17.456242    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:17.519769 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:17.519699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:17.537670 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:17.537638    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:30:17.904044 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:17.903997    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:18.072546 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:18.072498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:18.183178 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:18.183074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:19.104550 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:19.104515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:19.193703 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:19.193653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:19.279826 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:19.279795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:19.406838 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:19.406753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:19.865156 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:19.865093    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:19.937273 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:19.937233    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58" podUID="ea22da64-1220-47b1-94ad-3fd33ba00cdc" containerName="main" containerID="cri-o://1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af" gracePeriod=45
Sep 10 15:30:20.192859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.192834    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:30:20.261735 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.261701    2562 generic.go:356] "Generic (PLEG): container finished" podID="ea22da64-1220-47b1-94ad-3fd33ba00cdc" containerID="1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af" exitCode=0
Sep 10 15:30:20.261892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.261742    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"
Sep 10 15:30:20.261892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.261743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58" event={"ID":"ea22da64-1220-47b1-94ad-3fd33ba00cdc","Type":"ContainerDied","Data":"1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af"}
Sep 10 15:30:20.261892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.261882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58" event={"ID":"ea22da64-1220-47b1-94ad-3fd33ba00cdc","Type":"ContainerDied","Data":"5fa3fb4d3880170da605ff16f3bb85cd5c95c752c8ad34e8d1a0f8e148a3f4f8"}
Sep 10 15:30:20.262078 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.261917    2562 scope.go:117] "RemoveContainer" containerID="1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af"
Sep 10 15:30:20.266847 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.266816    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tls-certs\") pod \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") "
Sep 10 15:30:20.266949 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.266868    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tmp-dir\") pod \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") "
Sep 10 15:30:20.266949 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.266927    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qvnh\" (UniqueName: \"kubernetes.io/projected/ea22da64-1220-47b1-94ad-3fd33ba00cdc-kube-api-access-2qvnh\") pod \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") "
Sep 10 15:30:20.267076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.266951    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-dshm\") pod \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") "
Sep 10 15:30:20.267076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.266982    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-home\") pod \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") "
Sep 10 15:30:20.267076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.267034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-model-cache\") pod \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\" (UID: \"ea22da64-1220-47b1-94ad-3fd33ba00cdc\") "
Sep 10 15:30:20.267219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.267192    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea22da64-1220-47b1-94ad-3fd33ba00cdc" (UID: "ea22da64-1220-47b1-94ad-3fd33ba00cdc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.267330 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.267302    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-home" (OuterVolumeSpecName: "home") pod "ea22da64-1220-47b1-94ad-3fd33ba00cdc" (UID: "ea22da64-1220-47b1-94ad-3fd33ba00cdc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.267464 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.267376    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea22da64-1220-47b1-94ad-3fd33ba00cdc" (UID: "ea22da64-1220-47b1-94ad-3fd33ba00cdc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.267524 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.267497    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.267524 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.267515    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.267611 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.267525    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.269527 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.269501    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-dshm" (OuterVolumeSpecName: "dshm") pod "ea22da64-1220-47b1-94ad-3fd33ba00cdc" (UID: "ea22da64-1220-47b1-94ad-3fd33ba00cdc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.269636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.269573    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea22da64-1220-47b1-94ad-3fd33ba00cdc" (UID: "ea22da64-1220-47b1-94ad-3fd33ba00cdc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:30:20.269636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.269607    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea22da64-1220-47b1-94ad-3fd33ba00cdc-kube-api-access-2qvnh" (OuterVolumeSpecName: "kube-api-access-2qvnh") pod "ea22da64-1220-47b1-94ad-3fd33ba00cdc" (UID: "ea22da64-1220-47b1-94ad-3fd33ba00cdc"). InnerVolumeSpecName "kube-api-access-2qvnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:30:20.272218 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.272200    2562 scope.go:117] "RemoveContainer" containerID="1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af"
Sep 10 15:30:20.272545 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:30:20.272524    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af\": container with ID starting with 1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af not found: ID does not exist" containerID="1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af"
Sep 10 15:30:20.272603 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.272554    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af"} err="failed to get container status \"1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af\": rpc error: code = NotFound desc = could not find container \"1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af\": container with ID starting with 1e58523e63f4fdaad4eeee572a1d470dd28339866b56a5140ec64cdf529198af not found: ID does not exist"
Sep 10 15:30:20.368357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.368319    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea22da64-1220-47b1-94ad-3fd33ba00cdc-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.368357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.368349    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qvnh\" (UniqueName: \"kubernetes.io/projected/ea22da64-1220-47b1-94ad-3fd33ba00cdc-kube-api-access-2qvnh\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.368357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.368359    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea22da64-1220-47b1-94ad-3fd33ba00cdc-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.584063 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.584034    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"]
Sep 10 15:30:20.588017 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.587994    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-4d992b9c-kserve-5bc895d8f9-z9q58"]
Sep 10 15:30:20.636119 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.636086    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs" podUID="cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" containerName="main" containerID="cri-o://cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1" gracePeriod=45
Sep 10 15:30:20.873637 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.873612    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:30:20.974680 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.974639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tmp-dir\") pod \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") "
Sep 10 15:30:20.974838 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.974716    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-dshm\") pod \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") "
Sep 10 15:30:20.974838 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.974788    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-model-cache\") pod \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") "
Sep 10 15:30:20.974838 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.974826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q687w\" (UniqueName: \"kubernetes.io/projected/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-kube-api-access-q687w\") pod \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") "
Sep 10 15:30:20.974937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.974886    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tls-certs\") pod \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") "
Sep 10 15:30:20.974937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.974912    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-home\") pod \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\" (UID: \"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83\") "
Sep 10 15:30:20.974937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.974919    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" (UID: "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.975098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.975074    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-model-cache" (OuterVolumeSpecName: "model-cache") pod "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" (UID: "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.975232 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.975205    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-home" (OuterVolumeSpecName: "home") pod "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" (UID: "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.975305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.975296    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.975362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.975309    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.975362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.975318    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:20.976890 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.976866    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-dshm" (OuterVolumeSpecName: "dshm") pod "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" (UID: "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:30:20.976975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.976897    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" (UID: "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:30:20.976975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:20.976923    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-kube-api-access-q687w" (OuterVolumeSpecName: "kube-api-access-q687w") pod "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" (UID: "cb6e5eca-a37a-4b63-bad1-92a4fd89dd83"). InnerVolumeSpecName "kube-api-access-q687w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:30:21.076416 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.076378    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:21.076416 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.076411    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q687w\" (UniqueName: \"kubernetes.io/projected/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-kube-api-access-q687w\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:21.076699 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.076426    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:30:21.270083 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.269980    2562 generic.go:356] "Generic (PLEG): container finished" podID="cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" containerID="cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1" exitCode=0
Sep 10 15:30:21.270083 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.270026    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"
Sep 10 15:30:21.270083 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.270019    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs" event={"ID":"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83","Type":"ContainerDied","Data":"cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1"}
Sep 10 15:30:21.270663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.270151    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs" event={"ID":"cb6e5eca-a37a-4b63-bad1-92a4fd89dd83","Type":"ContainerDied","Data":"d9ab5902dab4dc24b641a27fc4a2968c60a2b51158e1a9d8c946126ceec05154"}
Sep 10 15:30:21.270663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.270197    2562 scope.go:117] "RemoveContainer" containerID="cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1"
Sep 10 15:30:21.279509 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.279488    2562 scope.go:117] "RemoveContainer" containerID="cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1"
Sep 10 15:30:21.279781 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:30:21.279761    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1\": container with ID starting with cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1 not found: ID does not exist" containerID="cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1"
Sep 10 15:30:21.279829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.279790    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1"} err="failed to get container status \"cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1\": rpc error: code = NotFound desc = could not find container \"cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1\": container with ID starting with cabbd76a7dfc04d11072dbc84acaa7813e58af0a88824883c0350ca42b4bc3f1 not found: ID does not exist"
Sep 10 15:30:21.294162 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.294137    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"]
Sep 10 15:30:21.303685 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.303659    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-4d992b9c-kserve-749c7fdc97-8q9gs"]
Sep 10 15:30:21.955817 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.955782    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" path="/var/lib/kubelet/pods/cb6e5eca-a37a-4b63-bad1-92a4fd89dd83/volumes"
Sep 10 15:30:21.956216 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:21.956202    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea22da64-1220-47b1-94ad-3fd33ba00cdc" path="/var/lib/kubelet/pods/ea22da64-1220-47b1-94ad-3fd33ba00cdc/volumes"
Sep 10 15:30:34.632724 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:34.632690    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:30:40.949940 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:40.949897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:45.717573 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:45.717531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:50.338291 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338223    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"]
Sep 10 15:30:50.338709 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338695    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" containerName="payload-pre-processing"
Sep 10 15:30:50.338709 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338710    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" containerName="payload-pre-processing"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338736    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea22da64-1220-47b1-94ad-3fd33ba00cdc" containerName="main"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338742    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea22da64-1220-47b1-94ad-3fd33ba00cdc" containerName="main"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338749    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce879c5-0d7e-48d6-905a-26625e4cb05c" containerName="cleanup"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338755    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce879c5-0d7e-48d6-905a-26625e4cb05c" containerName="cleanup"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338761    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" containerName="main"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338766    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" containerName="main"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338772    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ade48de-2f8e-451c-9ea9-a88fd53ff756" containerName="cleanup"
Sep 10 15:30:50.338799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338778    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ade48de-2f8e-451c-9ea9-a88fd53ff756" containerName="cleanup"
Sep 10 15:30:50.339040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338840    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea22da64-1220-47b1-94ad-3fd33ba00cdc" containerName="main"
Sep 10 15:30:50.339040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338849    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce879c5-0d7e-48d6-905a-26625e4cb05c" containerName="cleanup"
Sep 10 15:30:50.339040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338855    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb6e5eca-a37a-4b63-bad1-92a4fd89dd83" containerName="main"
Sep 10 15:30:50.339040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338861    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc6cad77-2d22-45a3-8f89-8e2d94ebeb8a" containerName="payload-pre-processing"
Sep 10 15:30:50.339040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.338870    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ade48de-2f8e-451c-9ea9-a88fd53ff756" containerName="cleanup"
Sep 10 15:30:50.344355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.344332    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.348274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.348242    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-50fe56\"/\"kube-root-ca.crt\""
Sep 10 15:30:50.348274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.348264    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-50fe56\"/\"test-model-50fe56-kserve-self-signed-certs\""
Sep 10 15:30:50.348527 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.348282    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-50fe56\"/\"default-dockercfg-wrbd7\""
Sep 10 15:30:50.348527 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.348248    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-50fe56\"/\"openshift-service-ca.crt\""
Sep 10 15:30:50.354054 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.354030    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"]
Sep 10 15:30:50.432502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.432475    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:30:50.462351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.462315    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c915b387-3bac-437c-af98-5dc357fcd2e0-tls-certs\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.462502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.462356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-tmp-dir\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.462502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.462383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-home\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.462629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.462537    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-model-cache\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.462629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.462596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-dshm\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.462726 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.462651    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9tz7\" (UniqueName: \"kubernetes.io/projected/c915b387-3bac-437c-af98-5dc357fcd2e0-kube-api-access-b9tz7\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.546128 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.546092    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:50.563277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-model-cache\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.563380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563292    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-dshm\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.563380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9tz7\" (UniqueName: \"kubernetes.io/projected/c915b387-3bac-437c-af98-5dc357fcd2e0-kube-api-access-b9tz7\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.563491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c915b387-3bac-437c-af98-5dc357fcd2e0-tls-certs\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.563491 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-tmp-dir\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.563610 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-home\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.563748 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-model-cache\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.564025 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.563998    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-tmp-dir\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.564163 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.564042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-home\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.566266 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.566238    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-dshm\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.566423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.566406    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c915b387-3bac-437c-af98-5dc357fcd2e0-tls-certs\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.572993 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.572970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9tz7\" (UniqueName: \"kubernetes.io/projected/c915b387-3bac-437c-af98-5dc357fcd2e0-kube-api-access-b9tz7\") pod \"test-model-50fe56-kserve-677976b599-xf5sk\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") " pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.679555 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.679469    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:30:50.958154 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:50.958128    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"]
Sep 10 15:30:50.960048 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:30:50.960025    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc915b387_3bac_437c_af98_5dc357fcd2e0.slice/crio-76b0f89d0852ba3f612496dcf23339d9fd52ceb3fb57f042a157c0c00204b551 WatchSource:0}: Error finding container 76b0f89d0852ba3f612496dcf23339d9fd52ceb3fb57f042a157c0c00204b551: Status 404 returned error can't find the container with id 76b0f89d0852ba3f612496dcf23339d9fd52ceb3fb57f042a157c0c00204b551
Sep 10 15:30:51.435373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:51.435335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk" event={"ID":"c915b387-3bac-437c-af98-5dc357fcd2e0","Type":"ContainerStarted","Data":"a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4"}
Sep 10 15:30:51.435796 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:51.435384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk" event={"ID":"c915b387-3bac-437c-af98-5dc357fcd2e0","Type":"ContainerStarted","Data":"76b0f89d0852ba3f612496dcf23339d9fd52ceb3fb57f042a157c0c00204b551"}
Sep 10 15:30:51.455440 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:51.455394    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk" podStartSLOduration=1.455376891 podStartE2EDuration="1.455376891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:30:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:30:51.453005312 +0000 UTC m=+891.988960874" watchObservedRunningTime="2026-09-10 15:30:51.455376891 +0000 UTC m=+891.991332453"
Sep 10 15:30:52.483024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:52.482933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:52.585151 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:52.585123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:52.676548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:52.676515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:59.931994 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:59.931966    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:30:59.932409 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:30:59.931969    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:31:00.679987 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:00.679954    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:31:00.680288 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:00.680267    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:31:00.692532 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:00.692502    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:31:01.497816 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:01.497781    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:31:02.241116 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:02.241080    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:31:04.755754 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:04.755717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:08.520205 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:08.520128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:14.908265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:14.908222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:15.201448 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:15.201318    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:15.301392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:15.301329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:18.336447 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:18.336407    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:31:22.220657 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:22.220620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:25.593656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:25.593622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:26.345865 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:26.345828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:32.849134 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:32.849089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:32.964849 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:32.964817    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.057988 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.057951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.159885 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.159748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.257806 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.257750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.354644 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.354609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.457339 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.457227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.542672 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.542633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.811224 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.811105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.884833 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.884802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:33.986105 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:33.986068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:35.428099 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:35.428051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:37.256641 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:37.256597    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:31:40.238344 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:40.238308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:40.337641 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:40.337602    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:40.516130 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:40.516040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:40.931095 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:40.931061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:41.275397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:41.275314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:41.368151 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:41.368119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:42.071962 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:42.071921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:42.154191 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:42.154141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:42.735502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:42.735440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:43.010899 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:43.010771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:43.103499 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:43.103431    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:43.206968 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:43.206905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:48.760943 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:48.760869    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:48.829064 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:48.829032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:49.022807 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:49.022711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:49.288679 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:49.288586    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:49.589800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:49.589739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:52.650999 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:52.650960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:52.747258 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:52.747222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:53.959963 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:53.959927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:54.060336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.060301    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:54.240425 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.240345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:54.331027 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.330987    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:31:54.449091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.449051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:54.554330 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.554244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:54.560276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.560249    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"]
Sep 10 15:31:54.564190 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.564167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:54.566971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.566943    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-83557387\""
Sep 10 15:31:54.567072 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.566970    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-83557387-dockercfg-48l7h\""
Sep 10 15:31:54.574671 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.574646    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"]
Sep 10 15:31:54.708915 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.708876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4tcq\" (UniqueName: \"kubernetes.io/projected/844a3693-a0ce-43a2-9df8-89abb8d94f39-kube-api-access-k4tcq\") pod \"payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:54.709090 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.709040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a3693-a0ce-43a2-9df8-89abb8d94f39-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:54.810165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.810072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a3693-a0ce-43a2-9df8-89abb8d94f39-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:54.810165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.810136    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4tcq\" (UniqueName: \"kubernetes.io/projected/844a3693-a0ce-43a2-9df8-89abb8d94f39-kube-api-access-k4tcq\") pod \"payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:54.810804 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.810783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a3693-a0ce-43a2-9df8-89abb8d94f39-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:54.818529 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.818502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4tcq\" (UniqueName: \"kubernetes.io/projected/844a3693-a0ce-43a2-9df8-89abb8d94f39-kube-api-access-k4tcq\") pod \"payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:54.875141 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:54.875100    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:55.147414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:55.147353    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"]
Sep 10 15:31:55.149924 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:31:55.149890    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844a3693_a0ce_43a2_9df8_89abb8d94f39.slice/crio-2f3d6cb31332481f34577ff9caf0c899ebc3c5bb584e1b71d41c5daeb5fd585c WatchSource:0}: Error finding container 2f3d6cb31332481f34577ff9caf0c899ebc3c5bb584e1b71d41c5daeb5fd585c: Status 404 returned error can't find the container with id 2f3d6cb31332481f34577ff9caf0c899ebc3c5bb584e1b71d41c5daeb5fd585c
Sep 10 15:31:55.769017 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:55.768976    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh" event={"ID":"844a3693-a0ce-43a2-9df8-89abb8d94f39","Type":"ContainerStarted","Data":"1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e"}
Sep 10 15:31:55.769212 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:55.769036    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh" event={"ID":"844a3693-a0ce-43a2-9df8-89abb8d94f39","Type":"ContainerStarted","Data":"2f3d6cb31332481f34577ff9caf0c899ebc3c5bb584e1b71d41c5daeb5fd585c"}
Sep 10 15:31:55.769212 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:55.769065    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:31:55.788509 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:31:55.788442    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh" podStartSLOduration=1.788425197 podStartE2EDuration="1.788425197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:31:55.786235978 +0000 UTC m=+956.322191566" watchObservedRunningTime="2026-09-10 15:31:55.788425197 +0000 UTC m=+956.324380759"
Sep 10 15:32:06.777643 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:06.777605    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:32:08.129975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:08.129939    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:32:11.033900 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.033859    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"]
Sep 10 15:32:11.037664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.037639    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.041505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.041375    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6c22f6\"/\"default-dockercfg-265rg\""
Sep 10 15:32:11.041505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.041416    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6c22f6\"/\"kube-root-ca.crt\""
Sep 10 15:32:11.041505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.041485    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6c22f6\"/\"openshift-service-ca.crt\""
Sep 10 15:32:11.041741 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.041485    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6c22f6\"/\"noten-model-6c22f6-kserve-self-signed-certs\""
Sep 10 15:32:11.050124 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.050097    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"]
Sep 10 15:32:11.091563 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.091530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:11.182535 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.182498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-model-cache\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.182787 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.182546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-home\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.182787 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.182605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-tmp-dir\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.182787 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.182648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-dshm\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.182787 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.182694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cltr5\" (UniqueName: \"kubernetes.io/projected/5dd605bd-bbff-4183-b777-076f2269fefd-kube-api-access-cltr5\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.182787 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.182739    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.283996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-model-cache\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-home\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-tmp-dir\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284383 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-dshm\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284383 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cltr5\" (UniqueName: \"kubernetes.io/projected/5dd605bd-bbff-4183-b777-076f2269fefd-kube-api-access-cltr5\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284383 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284283    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284572 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284479    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-model-cache\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284572 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284521    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-home\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.284572 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.284557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-tmp-dir\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.286424 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.286402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-dshm\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.286734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.286717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.292606 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.292581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cltr5\" (UniqueName: \"kubernetes.io/projected/5dd605bd-bbff-4183-b777-076f2269fefd-kube-api-access-cltr5\") pod \"noten-model-6c22f6-kserve-7854bbbb75-gfdw8\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") " pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.347663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.347630    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:11.620353 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.620326    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"]
Sep 10 15:32:11.622141 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:32:11.622109    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5dd605bd_bbff_4183_b777_076f2269fefd.slice/crio-800f82ec1bc3dedb3fc1e13237ffb4bd63274cc1aa0f05030dff9af9b099110e WatchSource:0}: Error finding container 800f82ec1bc3dedb3fc1e13237ffb4bd63274cc1aa0f05030dff9af9b099110e: Status 404 returned error can't find the container with id 800f82ec1bc3dedb3fc1e13237ffb4bd63274cc1aa0f05030dff9af9b099110e
Sep 10 15:32:11.854624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.854585    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8" event={"ID":"5dd605bd-bbff-4183-b777-076f2269fefd","Type":"ContainerStarted","Data":"948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26"}
Sep 10 15:32:11.854624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.854629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8" event={"ID":"5dd605bd-bbff-4183-b777-076f2269fefd","Type":"ContainerStarted","Data":"800f82ec1bc3dedb3fc1e13237ffb4bd63274cc1aa0f05030dff9af9b099110e"}
Sep 10 15:32:11.874515 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:11.874410    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8" podStartSLOduration=0.874394973 podStartE2EDuration="874.394973ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:32:11.872088259 +0000 UTC m=+972.408043834" watchObservedRunningTime="2026-09-10 15:32:11.874394973 +0000 UTC m=+972.410350535"
Sep 10 15:32:13.217109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:13.217072    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:17.567279 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:17.567238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:17.830759 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:17.830658    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:32:18.209397 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:18.209314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:18.299608 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:18.299576    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:18.629840 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:18.629803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:20.547123 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:20.547089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:21.348752 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:21.348710    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:21.348947 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:21.348787    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:21.361503 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:21.361475    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:21.918499 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:21.918470    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:22.908399 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:22.908365    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"]
Sep 10 15:32:22.927939 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:22.927906    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:23.133201 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:23.133170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:24.014733 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:24.014696    2562 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-self-signed-certs: secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:24.015206 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:24.014797    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs podName:5dd605bd-bbff-4183-b777-076f2269fefd nodeName:}" failed. No retries permitted until 2026-09-10 15:32:24.514772731 +0000 UTC m=+985.050728293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs") pod "noten-model-6c22f6-kserve-7854bbbb75-gfdw8" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd") : secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:24.037551 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:24.037517    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"]
Sep 10 15:32:24.054164 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:24.054134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:24.172469 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:24.172429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:24.380712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:24.380674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:24.481351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:24.481309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:24.520020 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:24.519983    2562 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-self-signed-certs: secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:24.520189 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:24.520062    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs podName:5dd605bd-bbff-4183-b777-076f2269fefd nodeName:}" failed. No retries permitted until 2026-09-10 15:32:25.520046741 +0000 UTC m=+986.056002282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs") pod "noten-model-6c22f6-kserve-7854bbbb75-gfdw8" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd") : secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:25.331991 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.331960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:25.431930 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.431896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:25.531468 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:25.531431    2562 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-self-signed-certs: secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:25.531655 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:25.531522    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs podName:5dd605bd-bbff-4183-b777-076f2269fefd nodeName:}" failed. No retries permitted until 2026-09-10 15:32:27.53150647 +0000 UTC m=+988.067462010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs") pod "noten-model-6c22f6-kserve-7854bbbb75-gfdw8" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd") : secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:25.684380 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.684297    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"]
Sep 10 15:32:25.688096 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.688079    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"
Sep 10 15:32:25.690978 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.690959    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:32:25.690978 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.690973    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:32:25.697874 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.697847    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"]
Sep 10 15:32:25.733274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.733234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b40f6ac-243d-4a89-baa2-20146a8e1c4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-83557387-jthwz\" (UID: \"7b40f6ac-243d-4a89-baa2-20146a8e1c4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"
Sep 10 15:32:25.803006 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.802970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:25.834361 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.834322    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b40f6ac-243d-4a89-baa2-20146a8e1c4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-83557387-jthwz\" (UID: \"7b40f6ac-243d-4a89-baa2-20146a8e1c4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"
Sep 10 15:32:25.835021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:25.834999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b40f6ac-243d-4a89-baa2-20146a8e1c4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-83557387-jthwz\" (UID: \"7b40f6ac-243d-4a89-baa2-20146a8e1c4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"
Sep 10 15:32:26.000225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:26.000136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"
Sep 10 15:32:26.272157 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:26.272131    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"]
Sep 10 15:32:26.274281 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:32:26.274252    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b40f6ac_243d_4a89_baa2_20146a8e1c4a.slice/crio-689b5724105509b9d82203d51e9009ce684b4b81a01edde2cc105509f2892461 WatchSource:0}: Error finding container 689b5724105509b9d82203d51e9009ce684b4b81a01edde2cc105509f2892461: Status 404 returned error can't find the container with id 689b5724105509b9d82203d51e9009ce684b4b81a01edde2cc105509f2892461
Sep 10 15:32:26.935756 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:26.935718    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz" event={"ID":"7b40f6ac-243d-4a89-baa2-20146a8e1c4a","Type":"ContainerStarted","Data":"ed3758fe90880afa34dc24d7aa9d25e92f2ca7341d15b423638eb2554102dd45"}
Sep 10 15:32:26.936215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:26.935769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz" event={"ID":"7b40f6ac-243d-4a89-baa2-20146a8e1c4a","Type":"ContainerStarted","Data":"689b5724105509b9d82203d51e9009ce684b4b81a01edde2cc105509f2892461"}
Sep 10 15:32:26.958941 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:26.958894    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz" podStartSLOduration=1.958875769 podStartE2EDuration="1.958875769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:32:26.955847988 +0000 UTC m=+987.491803573" watchObservedRunningTime="2026-09-10 15:32:26.958875769 +0000 UTC m=+987.494831332"
Sep 10 15:32:27.551833 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:27.551779    2562 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-self-signed-certs: secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:27.552034 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:27.551866    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs podName:5dd605bd-bbff-4183-b777-076f2269fefd nodeName:}" failed. No retries permitted until 2026-09-10 15:32:31.551850841 +0000 UTC m=+992.087806382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs") pod "noten-model-6c22f6-kserve-7854bbbb75-gfdw8" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd") : secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:27.942442 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:27.942403    2562 generic.go:356] "Generic (PLEG): container finished" podID="7b40f6ac-243d-4a89-baa2-20146a8e1c4a" containerID="ed3758fe90880afa34dc24d7aa9d25e92f2ca7341d15b423638eb2554102dd45" exitCode=0
Sep 10 15:32:27.942903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:27.942481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz" event={"ID":"7b40f6ac-243d-4a89-baa2-20146a8e1c4a","Type":"ContainerDied","Data":"ed3758fe90880afa34dc24d7aa9d25e92f2ca7341d15b423638eb2554102dd45"}
Sep 10 15:32:29.086811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.086787    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"
Sep 10 15:32:29.170080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.170033    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b40f6ac-243d-4a89-baa2-20146a8e1c4a-maas-api-service-ca\") pod \"7b40f6ac-243d-4a89-baa2-20146a8e1c4a\" (UID: \"7b40f6ac-243d-4a89-baa2-20146a8e1c4a\") "
Sep 10 15:32:29.170428 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.170404    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b40f6ac-243d-4a89-baa2-20146a8e1c4a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b40f6ac-243d-4a89-baa2-20146a8e1c4a" (UID: "7b40f6ac-243d-4a89-baa2-20146a8e1c4a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:32:29.271943 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.271858    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b40f6ac-243d-4a89-baa2-20146a8e1c4a-maas-api-service-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:29.555255 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.555169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:29.961691 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.961656    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"
Sep 10 15:32:29.962788 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.962757    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz" event={"ID":"7b40f6ac-243d-4a89-baa2-20146a8e1c4a","Type":"ContainerDied","Data":"689b5724105509b9d82203d51e9009ce684b4b81a01edde2cc105509f2892461"}
Sep 10 15:32:29.962938 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:29.962797    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="689b5724105509b9d82203d51e9009ce684b4b81a01edde2cc105509f2892461"
Sep 10 15:32:31.599044 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:31.599003    2562 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-self-signed-certs: secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:31.599490 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:31.599079    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs podName:5dd605bd-bbff-4183-b777-076f2269fefd nodeName:}" failed. No retries permitted until 2026-09-10 15:32:39.599065205 +0000 UTC m=+1000.135020745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs") pod "noten-model-6c22f6-kserve-7854bbbb75-gfdw8" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd") : secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:32.146607 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:32.146568    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:32:33.256132 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.256100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:33.877731 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.877691    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"]
Sep 10 15:32:33.883016 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.882985    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-229c42-29817570-zh258"]
Sep 10 15:32:33.885944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.885900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:33.921986 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.921951    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"]
Sep 10 15:32:33.922424 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.922395    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7" podUID="f221ef68-860d-4c17-a111-741551f0c43a" containerName="payload-pre-processing" containerID="cri-o://a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e" gracePeriod=30
Sep 10 15:32:33.962799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.962768    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ade48de-2f8e-451c-9ea9-a88fd53ff756" path="/var/lib/kubelet/pods/0ade48de-2f8e-451c-9ea9-a88fd53ff756/volumes"
Sep 10 15:32:33.985317 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:33.985289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:34.073684 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.073652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:34.164175 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.164135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:34.186736 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.186706    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:32:34.330918 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.330008    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nt8b\" (UniqueName: \"kubernetes.io/projected/f221ef68-860d-4c17-a111-741551f0c43a-kube-api-access-4nt8b\") pod \"f221ef68-860d-4c17-a111-741551f0c43a\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") "
Sep 10 15:32:34.330918 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.330126    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f221ef68-860d-4c17-a111-741551f0c43a-plugins-config-volume\") pod \"f221ef68-860d-4c17-a111-741551f0c43a\" (UID: \"f221ef68-860d-4c17-a111-741551f0c43a\") "
Sep 10 15:32:34.331937 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.331613    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f221ef68-860d-4c17-a111-741551f0c43a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f221ef68-860d-4c17-a111-741551f0c43a" (UID: "f221ef68-860d-4c17-a111-741551f0c43a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:32:34.335614 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.334247    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f221ef68-860d-4c17-a111-741551f0c43a-kube-api-access-4nt8b" (OuterVolumeSpecName: "kube-api-access-4nt8b") pod "f221ef68-860d-4c17-a111-741551f0c43a" (UID: "f221ef68-860d-4c17-a111-741551f0c43a"). InnerVolumeSpecName "kube-api-access-4nt8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:32:34.431559 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.431440    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nt8b\" (UniqueName: \"kubernetes.io/projected/f221ef68-860d-4c17-a111-741551f0c43a-kube-api-access-4nt8b\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:34.431559 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.431501    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f221ef68-860d-4c17-a111-741551f0c43a-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:34.436882 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.436856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:34.989154 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.989112    2562 generic.go:356] "Generic (PLEG): container finished" podID="f221ef68-860d-4c17-a111-741551f0c43a" containerID="a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e" exitCode=0
Sep 10 15:32:34.989362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.989177    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"
Sep 10 15:32:34.989362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.989201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7" event={"ID":"f221ef68-860d-4c17-a111-741551f0c43a","Type":"ContainerDied","Data":"a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e"}
Sep 10 15:32:34.989362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.989262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7" event={"ID":"f221ef68-860d-4c17-a111-741551f0c43a","Type":"ContainerDied","Data":"e4fae4747666fef5a3a5fe4f1cbfeae00bf69960a84cafe138ed84739517c50e"}
Sep 10 15:32:34.989362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.989285    2562 scope.go:117] "RemoveContainer" containerID="a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e"
Sep 10 15:32:34.998545 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.998522    2562 scope.go:117] "RemoveContainer" containerID="a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e"
Sep 10 15:32:34.998839 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:34.998821    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e\": container with ID starting with a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e not found: ID does not exist" containerID="a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e"
Sep 10 15:32:34.998907 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:34.998846    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e"} err="failed to get container status \"a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e\": rpc error: code = NotFound desc = could not find container \"a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e\": container with ID starting with a2a35a6b8dfe0d06a75c2135f99994ebecabcc5bb77099d36f979617e1e7032e not found: ID does not exist"
Sep 10 15:32:35.012098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:35.012060    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"]
Sep 10 15:32:35.016170 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:35.016144    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-229c42-77b65db46-bkzp7"]
Sep 10 15:32:35.954613 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:35.954579    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:35.955690 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:35.955669    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f221ef68-860d-4c17-a111-741551f0c43a" path="/var/lib/kubelet/pods/f221ef68-860d-4c17-a111-741551f0c43a/volumes"
Sep 10 15:32:36.536796 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.536762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:36.579137 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.579087    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"]
Sep 10 15:32:36.579708 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.579675    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh" podUID="844a3693-a0ce-43a2-9df8-89abb8d94f39" containerName="payload-pre-processing" containerID="cri-o://1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e" gracePeriod=30
Sep 10 15:32:36.637215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.637183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:36.740043 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.740005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:36.774202 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.774167    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"]
Sep 10 15:32:36.778344 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.778312    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83557387-jthwz"]
Sep 10 15:32:36.826273 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.826246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:36.832960 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.832936    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:32:36.959888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.959853    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a3693-a0ce-43a2-9df8-89abb8d94f39-plugins-config-volume\") pod \"844a3693-a0ce-43a2-9df8-89abb8d94f39\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") "
Sep 10 15:32:36.960367 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.959932    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4tcq\" (UniqueName: \"kubernetes.io/projected/844a3693-a0ce-43a2-9df8-89abb8d94f39-kube-api-access-k4tcq\") pod \"844a3693-a0ce-43a2-9df8-89abb8d94f39\" (UID: \"844a3693-a0ce-43a2-9df8-89abb8d94f39\") "
Sep 10 15:32:36.960367 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.960333    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/844a3693-a0ce-43a2-9df8-89abb8d94f39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "844a3693-a0ce-43a2-9df8-89abb8d94f39" (UID: "844a3693-a0ce-43a2-9df8-89abb8d94f39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:32:36.962058 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:36.962031    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844a3693-a0ce-43a2-9df8-89abb8d94f39-kube-api-access-k4tcq" (OuterVolumeSpecName: "kube-api-access-k4tcq") pod "844a3693-a0ce-43a2-9df8-89abb8d94f39" (UID: "844a3693-a0ce-43a2-9df8-89abb8d94f39"). InnerVolumeSpecName "kube-api-access-k4tcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:32:37.001638 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.001607    2562 generic.go:356] "Generic (PLEG): container finished" podID="844a3693-a0ce-43a2-9df8-89abb8d94f39" containerID="1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e" exitCode=0
Sep 10 15:32:37.001778 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.001654    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"
Sep 10 15:32:37.001778 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.001666    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh" event={"ID":"844a3693-a0ce-43a2-9df8-89abb8d94f39","Type":"ContainerDied","Data":"1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e"}
Sep 10 15:32:37.001778 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.001708    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh" event={"ID":"844a3693-a0ce-43a2-9df8-89abb8d94f39","Type":"ContainerDied","Data":"2f3d6cb31332481f34577ff9caf0c899ebc3c5bb584e1b71d41c5daeb5fd585c"}
Sep 10 15:32:37.001778 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.001730    2562 scope.go:117] "RemoveContainer" containerID="1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e"
Sep 10 15:32:37.010895 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.010876    2562 scope.go:117] "RemoveContainer" containerID="1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e"
Sep 10 15:32:37.011185 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:37.011165    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e\": container with ID starting with 1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e not found: ID does not exist" containerID="1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e"
Sep 10 15:32:37.011247 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.011193    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e"} err="failed to get container status \"1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e\": rpc error: code = NotFound desc = could not find container \"1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e\": container with ID starting with 1dd7a0565e9d699d4d68db1711b8410ccaf482edee6d7cb9a8f268f73df2314e not found: ID does not exist"
Sep 10 15:32:37.023705 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.023680    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"]
Sep 10 15:32:37.029510 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.029485    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh"]
Sep 10 15:32:37.061760 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.061691    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a3693-a0ce-43a2-9df8-89abb8d94f39-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:37.061760 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.061715    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4tcq\" (UniqueName: \"kubernetes.io/projected/844a3693-a0ce-43a2-9df8-89abb8d94f39-kube-api-access-k4tcq\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:37.129157 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.129124    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:37.775365 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.775318    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-83557387-67fb84d86-8llgh" podUID="844a3693-a0ce-43a2-9df8-89abb8d94f39" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.48:9004: i/o timeout"
Sep 10 15:32:37.929817 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.929776    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk" podUID="c915b387-3bac-437c-af98-5dc357fcd2e0" containerName="main" containerID="cri-o://a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4" gracePeriod=45
Sep 10 15:32:37.956013 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.955979    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b40f6ac-243d-4a89-baa2-20146a8e1c4a" path="/var/lib/kubelet/pods/7b40f6ac-243d-4a89-baa2-20146a8e1c4a/volumes"
Sep 10 15:32:37.956369 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:37.956351    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="844a3693-a0ce-43a2-9df8-89abb8d94f39" path="/var/lib/kubelet/pods/844a3693-a0ce-43a2-9df8-89abb8d94f39/volumes"
Sep 10 15:32:38.189894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.189869    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:32:38.375577 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.375539    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9tz7\" (UniqueName: \"kubernetes.io/projected/c915b387-3bac-437c-af98-5dc357fcd2e0-kube-api-access-b9tz7\") pod \"c915b387-3bac-437c-af98-5dc357fcd2e0\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") "
Sep 10 15:32:38.375768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.375608    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-dshm\") pod \"c915b387-3bac-437c-af98-5dc357fcd2e0\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") "
Sep 10 15:32:38.375768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.375630    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c915b387-3bac-437c-af98-5dc357fcd2e0-tls-certs\") pod \"c915b387-3bac-437c-af98-5dc357fcd2e0\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") "
Sep 10 15:32:38.375768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.375670    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-model-cache\") pod \"c915b387-3bac-437c-af98-5dc357fcd2e0\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") "
Sep 10 15:32:38.375768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.375689    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-home\") pod \"c915b387-3bac-437c-af98-5dc357fcd2e0\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") "
Sep 10 15:32:38.375989 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.375836    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-tmp-dir\") pod \"c915b387-3bac-437c-af98-5dc357fcd2e0\" (UID: \"c915b387-3bac-437c-af98-5dc357fcd2e0\") "
Sep 10 15:32:38.375989 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.375953    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "c915b387-3bac-437c-af98-5dc357fcd2e0" (UID: "c915b387-3bac-437c-af98-5dc357fcd2e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:38.376080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.376016    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-home" (OuterVolumeSpecName: "home") pod "c915b387-3bac-437c-af98-5dc357fcd2e0" (UID: "c915b387-3bac-437c-af98-5dc357fcd2e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:38.376080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.376061    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c915b387-3bac-437c-af98-5dc357fcd2e0" (UID: "c915b387-3bac-437c-af98-5dc357fcd2e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:38.376237 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.376224    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:38.376276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.376240    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:38.376276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.376248    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:38.377907 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.377883    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c915b387-3bac-437c-af98-5dc357fcd2e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c915b387-3bac-437c-af98-5dc357fcd2e0" (UID: "c915b387-3bac-437c-af98-5dc357fcd2e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:32:38.378151 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.378132    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-dshm" (OuterVolumeSpecName: "dshm") pod "c915b387-3bac-437c-af98-5dc357fcd2e0" (UID: "c915b387-3bac-437c-af98-5dc357fcd2e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:38.378240 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.378216    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c915b387-3bac-437c-af98-5dc357fcd2e0-kube-api-access-b9tz7" (OuterVolumeSpecName: "kube-api-access-b9tz7") pod "c915b387-3bac-437c-af98-5dc357fcd2e0" (UID: "c915b387-3bac-437c-af98-5dc357fcd2e0"). InnerVolumeSpecName "kube-api-access-b9tz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:32:38.476981 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.476942    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9tz7\" (UniqueName: \"kubernetes.io/projected/c915b387-3bac-437c-af98-5dc357fcd2e0-kube-api-access-b9tz7\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:38.476981 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.476975    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c915b387-3bac-437c-af98-5dc357fcd2e0-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:38.476981 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:38.476990    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c915b387-3bac-437c-af98-5dc357fcd2e0-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:39.017614 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.017578    2562 generic.go:356] "Generic (PLEG): container finished" podID="c915b387-3bac-437c-af98-5dc357fcd2e0" containerID="a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4" exitCode=0
Sep 10 15:32:39.017801 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.017630    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"
Sep 10 15:32:39.017801 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.017633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk" event={"ID":"c915b387-3bac-437c-af98-5dc357fcd2e0","Type":"ContainerDied","Data":"a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4"}
Sep 10 15:32:39.017801 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.017735    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk" event={"ID":"c915b387-3bac-437c-af98-5dc357fcd2e0","Type":"ContainerDied","Data":"76b0f89d0852ba3f612496dcf23339d9fd52ceb3fb57f042a157c0c00204b551"}
Sep 10 15:32:39.017801 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.017751    2562 scope.go:117] "RemoveContainer" containerID="a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4"
Sep 10 15:32:39.026952 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.026932    2562 scope.go:117] "RemoveContainer" containerID="a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4"
Sep 10 15:32:39.027241 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:39.027222    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4\": container with ID starting with a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4 not found: ID does not exist" containerID="a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4"
Sep 10 15:32:39.027294 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.027249    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4"} err="failed to get container status \"a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4\": rpc error: code = NotFound desc = could not find container \"a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4\": container with ID starting with a5b1433784c5145612c4ac35d96df67ee48c8cb6cd9884c38b10d412f562f0f4 not found: ID does not exist"
Sep 10 15:32:39.039631 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.039605    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"]
Sep 10 15:32:39.043787 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.043763    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-50fe56/test-model-50fe56-kserve-677976b599-xf5sk"]
Sep 10 15:32:39.689917 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:39.689876    2562 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-self-signed-certs: secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:39.690389 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:39.689961    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs podName:5dd605bd-bbff-4183-b777-076f2269fefd nodeName:}" failed. No retries permitted until 2026-09-10 15:32:55.689940708 +0000 UTC m=+1016.225896248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs") pod "noten-model-6c22f6-kserve-7854bbbb75-gfdw8" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd") : secret "noten-model-6c22f6-kserve-self-signed-certs" not found
Sep 10 15:32:39.943884 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.943789    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8" podUID="5dd605bd-bbff-4183-b777-076f2269fefd" containerName="main" containerID="cri-o://948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26" gracePeriod=45
Sep 10 15:32:39.957127 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:39.957094    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c915b387-3bac-437c-af98-5dc357fcd2e0" path="/var/lib/kubelet/pods/c915b387-3bac-437c-af98-5dc357fcd2e0/volumes"
Sep 10 15:32:40.205130 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.205072    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:40.398616 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.398580    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-dshm\") pod \"5dd605bd-bbff-4183-b777-076f2269fefd\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") "
Sep 10 15:32:40.398616 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.398622    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cltr5\" (UniqueName: \"kubernetes.io/projected/5dd605bd-bbff-4183-b777-076f2269fefd-kube-api-access-cltr5\") pod \"5dd605bd-bbff-4183-b777-076f2269fefd\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") "
Sep 10 15:32:40.398886 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.398665    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-tmp-dir\") pod \"5dd605bd-bbff-4183-b777-076f2269fefd\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") "
Sep 10 15:32:40.398886 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.398774    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-home\") pod \"5dd605bd-bbff-4183-b777-076f2269fefd\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") "
Sep 10 15:32:40.398886 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.398858    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-model-cache\") pod \"5dd605bd-bbff-4183-b777-076f2269fefd\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") "
Sep 10 15:32:40.399036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.398893    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs\") pod \"5dd605bd-bbff-4183-b777-076f2269fefd\" (UID: \"5dd605bd-bbff-4183-b777-076f2269fefd\") "
Sep 10 15:32:40.399036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.398919    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5dd605bd-bbff-4183-b777-076f2269fefd" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:40.399036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.399014    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-home" (OuterVolumeSpecName: "home") pod "5dd605bd-bbff-4183-b777-076f2269fefd" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:40.399196 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.399104    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-model-cache" (OuterVolumeSpecName: "model-cache") pod "5dd605bd-bbff-4183-b777-076f2269fefd" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:40.399281 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.399266    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:40.399343 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.399283    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:40.399343 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.399291    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:40.400940 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.400911    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-dshm" (OuterVolumeSpecName: "dshm") pod "5dd605bd-bbff-4183-b777-076f2269fefd" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:32:40.400940 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.400912    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd605bd-bbff-4183-b777-076f2269fefd-kube-api-access-cltr5" (OuterVolumeSpecName: "kube-api-access-cltr5") pod "5dd605bd-bbff-4183-b777-076f2269fefd" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd"). InnerVolumeSpecName "kube-api-access-cltr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:32:40.401083 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.400958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5dd605bd-bbff-4183-b777-076f2269fefd" (UID: "5dd605bd-bbff-4183-b777-076f2269fefd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:32:40.500660 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.500573    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dd605bd-bbff-4183-b777-076f2269fefd-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:40.500660 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.500607    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cltr5\" (UniqueName: \"kubernetes.io/projected/5dd605bd-bbff-4183-b777-076f2269fefd-kube-api-access-cltr5\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:40.500660 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.500617    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dd605bd-bbff-4183-b777-076f2269fefd-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:40.770619 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.770532    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:40.870521 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:40.870487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:41.028422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.028338    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:41.030961 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.030932    2562 generic.go:356] "Generic (PLEG): container finished" podID="5dd605bd-bbff-4183-b777-076f2269fefd" containerID="948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26" exitCode=0
Sep 10 15:32:41.031109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.030988    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"
Sep 10 15:32:41.031109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.030997    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8" event={"ID":"5dd605bd-bbff-4183-b777-076f2269fefd","Type":"ContainerDied","Data":"948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26"}
Sep 10 15:32:41.031109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.031028    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8" event={"ID":"5dd605bd-bbff-4183-b777-076f2269fefd","Type":"ContainerDied","Data":"800f82ec1bc3dedb3fc1e13237ffb4bd63274cc1aa0f05030dff9af9b099110e"}
Sep 10 15:32:41.031109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.031044    2562 scope.go:117] "RemoveContainer" containerID="948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26"
Sep 10 15:32:41.040543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.040524    2562 scope.go:117] "RemoveContainer" containerID="948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26"
Sep 10 15:32:41.040806 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:32:41.040788    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26\": container with ID starting with 948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26 not found: ID does not exist" containerID="948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26"
Sep 10 15:32:41.040869 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.040819    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26"} err="failed to get container status \"948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26\": rpc error: code = NotFound desc = could not find container \"948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26\": container with ID starting with 948f0dba0cd5e4edb735f7ef88f3c97761fdc35ea7b6f04f49300a78cb3ddf26 not found: ID does not exist"
Sep 10 15:32:41.052340 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.052311    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"]
Sep 10 15:32:41.057986 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.057963    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-6c22f6/noten-model-6c22f6-kserve-7854bbbb75-gfdw8"]
Sep 10 15:32:41.463486 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.463418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:41.539368 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.539326    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:32:41.955796 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:41.955762    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd605bd-bbff-4183-b777-076f2269fefd" path="/var/lib/kubelet/pods/5dd605bd-bbff-4183-b777-076f2269fefd/volumes"
Sep 10 15:32:43.595710 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:43.595673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:43.669054 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:43.669019    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:43.883181 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:43.883088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:44.167654 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:44.167559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:44.290804 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:44.290771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:44.430373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:44.430280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:50.435617 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:50.435581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:50.543003 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:50.542972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:50.636389 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:50.636354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:50.826026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:50.825987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:51.873718 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:51.873678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:51.904580 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:51.904544    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"]
Sep 10 15:32:51.904894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:51.904846    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm" podUID="9225f8a5-a47a-4757-adb1-8459a0d0b0c7" containerName="payload-processing" containerID="cri-o://a5b5bbb5b29b117887d67ea303689da0224fe81f7cbcd1d8363e53b3e23f40f5" gracePeriod=30
Sep 10 15:32:51.914951 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:51.914925    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"]
Sep 10 15:32:51.915343 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:51.915311    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7" podUID="5eb3962f-05b7-476b-99f5-10eb75537df6" containerName="payload-pre-processing" containerID="cri-o://49c4ec8677b1efd2791a348f7426f9d5ab18897eed3f75c053b920cdbf7632a4" gracePeriod=30
Sep 10 15:32:51.955279 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:51.955237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:52.030820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.030785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:52.098464 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.098423    2562 generic.go:356] "Generic (PLEG): container finished" podID="5eb3962f-05b7-476b-99f5-10eb75537df6" containerID="49c4ec8677b1efd2791a348f7426f9d5ab18897eed3f75c053b920cdbf7632a4" exitCode=0
Sep 10 15:32:52.098629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.098492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7" event={"ID":"5eb3962f-05b7-476b-99f5-10eb75537df6","Type":"ContainerDied","Data":"49c4ec8677b1efd2791a348f7426f9d5ab18897eed3f75c053b920cdbf7632a4"}
Sep 10 15:32:52.100587 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.100551    2562 generic.go:356] "Generic (PLEG): container finished" podID="9225f8a5-a47a-4757-adb1-8459a0d0b0c7" containerID="a5b5bbb5b29b117887d67ea303689da0224fe81f7cbcd1d8363e53b3e23f40f5" exitCode=0
Sep 10 15:32:52.100719 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.100630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm" event={"ID":"9225f8a5-a47a-4757-adb1-8459a0d0b0c7","Type":"ContainerDied","Data":"a5b5bbb5b29b117887d67ea303689da0224fe81f7cbcd1d8363e53b3e23f40f5"}
Sep 10 15:32:52.105924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.105891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:52.166592 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.166568    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:32:52.180440 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.180297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:52.191693 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.191664    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:32:52.222371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.222346    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-plugins-config-volume\") pod \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") "
Sep 10 15:32:52.222570 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.222434    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgq68\" (UniqueName: \"kubernetes.io/projected/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-kube-api-access-vgq68\") pod \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\" (UID: \"9225f8a5-a47a-4757-adb1-8459a0d0b0c7\") "
Sep 10 15:32:52.222791 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.222762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9225f8a5-a47a-4757-adb1-8459a0d0b0c7" (UID: "9225f8a5-a47a-4757-adb1-8459a0d0b0c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:32:52.224383 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.224365    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-kube-api-access-vgq68" (OuterVolumeSpecName: "kube-api-access-vgq68") pod "9225f8a5-a47a-4757-adb1-8459a0d0b0c7" (UID: "9225f8a5-a47a-4757-adb1-8459a0d0b0c7"). InnerVolumeSpecName "kube-api-access-vgq68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:32:52.323112 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.323073    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9wxf\" (UniqueName: \"kubernetes.io/projected/5eb3962f-05b7-476b-99f5-10eb75537df6-kube-api-access-t9wxf\") pod \"5eb3962f-05b7-476b-99f5-10eb75537df6\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") "
Sep 10 15:32:52.323287 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.323199    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eb3962f-05b7-476b-99f5-10eb75537df6-plugins-config-volume\") pod \"5eb3962f-05b7-476b-99f5-10eb75537df6\" (UID: \"5eb3962f-05b7-476b-99f5-10eb75537df6\") "
Sep 10 15:32:52.323584 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.323551    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb3962f-05b7-476b-99f5-10eb75537df6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5eb3962f-05b7-476b-99f5-10eb75537df6" (UID: "5eb3962f-05b7-476b-99f5-10eb75537df6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:32:52.323697 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.323613    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:52.323697 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.323628    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgq68\" (UniqueName: \"kubernetes.io/projected/9225f8a5-a47a-4757-adb1-8459a0d0b0c7-kube-api-access-vgq68\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:52.325215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.325197    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb3962f-05b7-476b-99f5-10eb75537df6-kube-api-access-t9wxf" (OuterVolumeSpecName: "kube-api-access-t9wxf") pod "5eb3962f-05b7-476b-99f5-10eb75537df6" (UID: "5eb3962f-05b7-476b-99f5-10eb75537df6"). InnerVolumeSpecName "kube-api-access-t9wxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:32:52.425107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.425011    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9wxf\" (UniqueName: \"kubernetes.io/projected/5eb3962f-05b7-476b-99f5-10eb75537df6-kube-api-access-t9wxf\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:52.425107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.425043    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eb3962f-05b7-476b-99f5-10eb75537df6-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:32:52.451242 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:52.451203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:53.106788 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.106756    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"
Sep 10 15:32:53.107268 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.106791    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7" event={"ID":"5eb3962f-05b7-476b-99f5-10eb75537df6","Type":"ContainerDied","Data":"14fa9602e0929e10195c618e81ae02882eb1e194a4d7a2cbc0ea0ad827d8d08f"}
Sep 10 15:32:53.107268 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.106846    2562 scope.go:117] "RemoveContainer" containerID="49c4ec8677b1efd2791a348f7426f9d5ab18897eed3f75c053b920cdbf7632a4"
Sep 10 15:32:53.108268 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.108211    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm" event={"ID":"9225f8a5-a47a-4757-adb1-8459a0d0b0c7","Type":"ContainerDied","Data":"721e677d9692950dbcd52cbae27dc94892b06aaf5f5ac08d2f9217017922b614"}
Sep 10 15:32:53.108268 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.108237    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"
Sep 10 15:32:53.117322 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.117302    2562 scope.go:117] "RemoveContainer" containerID="a5b5bbb5b29b117887d67ea303689da0224fe81f7cbcd1d8363e53b3e23f40f5"
Sep 10 15:32:53.148673 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.148647    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"]
Sep 10 15:32:53.159717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.159689    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-50fe56-6764c977fd-8bkv7"]
Sep 10 15:32:53.174363 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.174298    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"]
Sep 10 15:32:53.182094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.182069    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-50fe56-5759f757c-q6mdm"]
Sep 10 15:32:53.955317 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.955286    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eb3962f-05b7-476b-99f5-10eb75537df6" path="/var/lib/kubelet/pods/5eb3962f-05b7-476b-99f5-10eb75537df6/volumes"
Sep 10 15:32:53.955706 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:53.955691    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9225f8a5-a47a-4757-adb1-8459a0d0b0c7" path="/var/lib/kubelet/pods/9225f8a5-a47a-4757-adb1-8459a0d0b0c7/volumes"
Sep 10 15:32:54.534522 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:54.534479    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:32:56.991750 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:56.991714    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:57.091333 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:57.091295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:57.871208 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:57.871163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:58.167965 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:58.167886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:58.446215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:58.446135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:59.045094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:59.045055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:59.143612 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:59.143578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:59.295785 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:59.295700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:59.816502 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:32:59.816445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:00.211576 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:00.211487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:00.903400 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:00.903362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:01.329811 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:01.329774    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:01.434913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:01.434878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:02.670614 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:02.670580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:02.749194 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:02.749165    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:03.136226 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.136190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:03.234167 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.234127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:03.275305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275274    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"]
Sep 10 15:33:03.275739 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275724    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844a3693-a0ce-43a2-9df8-89abb8d94f39" containerName="payload-pre-processing"
Sep 10 15:33:03.275790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275741    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="844a3693-a0ce-43a2-9df8-89abb8d94f39" containerName="payload-pre-processing"
Sep 10 15:33:03.275790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275754    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c915b387-3bac-437c-af98-5dc357fcd2e0" containerName="main"
Sep 10 15:33:03.275790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275759    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c915b387-3bac-437c-af98-5dc357fcd2e0" containerName="main"
Sep 10 15:33:03.275790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275767    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b40f6ac-243d-4a89-baa2-20146a8e1c4a" containerName="revoke-keys"
Sep 10 15:33:03.275790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275772    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b40f6ac-243d-4a89-baa2-20146a8e1c4a" containerName="revoke-keys"
Sep 10 15:33:03.275790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275786    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f221ef68-860d-4c17-a111-741551f0c43a" containerName="payload-pre-processing"
Sep 10 15:33:03.275790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275791    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f221ef68-860d-4c17-a111-741551f0c43a" containerName="payload-pre-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275797    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd605bd-bbff-4183-b777-076f2269fefd" containerName="main"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275802    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd605bd-bbff-4183-b777-076f2269fefd" containerName="main"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275819    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb3962f-05b7-476b-99f5-10eb75537df6" containerName="payload-pre-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275825    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb3962f-05b7-476b-99f5-10eb75537df6" containerName="payload-pre-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275831    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9225f8a5-a47a-4757-adb1-8459a0d0b0c7" containerName="payload-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275837    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9225f8a5-a47a-4757-adb1-8459a0d0b0c7" containerName="payload-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275893    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="844a3693-a0ce-43a2-9df8-89abb8d94f39" containerName="payload-pre-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275902    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9225f8a5-a47a-4757-adb1-8459a0d0b0c7" containerName="payload-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275908    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb3962f-05b7-476b-99f5-10eb75537df6" containerName="payload-pre-processing"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275916    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c915b387-3bac-437c-af98-5dc357fcd2e0" containerName="main"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275921    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd605bd-bbff-4183-b777-076f2269fefd" containerName="main"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275926    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b40f6ac-243d-4a89-baa2-20146a8e1c4a" containerName="revoke-keys"
Sep 10 15:33:03.276000 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.275932    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f221ef68-860d-4c17-a111-741551f0c43a" containerName="payload-pre-processing"
Sep 10 15:33:03.280298 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.280281    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.282962 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.282939    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-5fe45b-dockercfg-9wbxk\""
Sep 10 15:33:03.283059 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.282989    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-5fe45b\""
Sep 10 15:33:03.288417 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.288389    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"]
Sep 10 15:33:03.447775 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.447683    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d636ff7a-a331-4cf2-b930-8013fce2d858-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.447775 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.447752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4z4l\" (UniqueName: \"kubernetes.io/projected/d636ff7a-a331-4cf2-b930-8013fce2d858-kube-api-access-r4z4l\") pod \"payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.548235 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.548200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d636ff7a-a331-4cf2-b930-8013fce2d858-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.548408 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.548261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4z4l\" (UniqueName: \"kubernetes.io/projected/d636ff7a-a331-4cf2-b930-8013fce2d858-kube-api-access-r4z4l\") pod \"payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.548932 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.548913    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d636ff7a-a331-4cf2-b930-8013fce2d858-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.556879 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.556847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4z4l\" (UniqueName: \"kubernetes.io/projected/d636ff7a-a331-4cf2-b930-8013fce2d858-kube-api-access-r4z4l\") pod \"payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.591854 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.591813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:03.863860 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:03.863797    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"]
Sep 10 15:33:03.866125 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:33:03.866102    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd636ff7a_a331_4cf2_b930_8013fce2d858.slice/crio-c7e5a6482acbec336906ba1dfc537ebe3f6a8558d6eb7d3948e121df045ba8e4 WatchSource:0}: Error finding container c7e5a6482acbec336906ba1dfc537ebe3f6a8558d6eb7d3948e121df045ba8e4: Status 404 returned error can't find the container with id c7e5a6482acbec336906ba1dfc537ebe3f6a8558d6eb7d3948e121df045ba8e4
Sep 10 15:33:04.170115 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:04.170037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp" event={"ID":"d636ff7a-a331-4cf2-b930-8013fce2d858","Type":"ContainerStarted","Data":"19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53"}
Sep 10 15:33:04.170115 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:04.170085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp" event={"ID":"d636ff7a-a331-4cf2-b930-8013fce2d858","Type":"ContainerStarted","Data":"c7e5a6482acbec336906ba1dfc537ebe3f6a8558d6eb7d3948e121df045ba8e4"}
Sep 10 15:33:04.170306 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:04.170201    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:04.191050 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:04.190998    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp" podStartSLOduration=1.190978393 podStartE2EDuration="1.190978393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:04.189069942 +0000 UTC m=+1024.725025597" watchObservedRunningTime="2026-09-10 15:33:04.190978393 +0000 UTC m=+1024.726933957"
Sep 10 15:33:05.848687 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:05.848653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:33:05.917161 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:05.917122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:06.011277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:06.011234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:07.126045 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.126005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:07.226393 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.226359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:07.631662 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.631627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:07.730888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.730856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:07.836985 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.836944    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"]
Sep 10 15:33:07.840683 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.840659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:07.843219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.843200    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-62d2f7-dockercfg-9qxl6\""
Sep 10 15:33:07.843327 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.843200    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-62d2f7\""
Sep 10 15:33:07.853974 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.853947    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"]
Sep 10 15:33:07.995342 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.995250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfllq\" (UniqueName: \"kubernetes.io/projected/1ba3a57f-92e5-4de4-acc1-277d1574bffe-kube-api-access-lfllq\") pod \"payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:07.995342 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:07.995293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ba3a57f-92e5-4de4-acc1-277d1574bffe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:08.096094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:08.096053    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfllq\" (UniqueName: \"kubernetes.io/projected/1ba3a57f-92e5-4de4-acc1-277d1574bffe-kube-api-access-lfllq\") pod \"payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:08.096094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:08.096098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ba3a57f-92e5-4de4-acc1-277d1574bffe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:08.096763 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:08.096743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ba3a57f-92e5-4de4-acc1-277d1574bffe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:08.105240 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:08.105216    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfllq\" (UniqueName: \"kubernetes.io/projected/1ba3a57f-92e5-4de4-acc1-277d1574bffe-kube-api-access-lfllq\") pod \"payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:08.154238 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:08.154198    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:08.204668 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:08.204420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:08.429976 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:08.429946    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"]
Sep 10 15:33:08.432706 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:33:08.432672    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ba3a57f_92e5_4de4_acc1_277d1574bffe.slice/crio-c4ffbf61bf00b2de6a98482351d4832e2d46822237b6b9291883ecee131c7f56 WatchSource:0}: Error finding container c4ffbf61bf00b2de6a98482351d4832e2d46822237b6b9291883ecee131c7f56: Status 404 returned error can't find the container with id c4ffbf61bf00b2de6a98482351d4832e2d46822237b6b9291883ecee131c7f56
Sep 10 15:33:09.200837 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:09.200803    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5" event={"ID":"1ba3a57f-92e5-4de4-acc1-277d1574bffe","Type":"ContainerStarted","Data":"d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7"}
Sep 10 15:33:09.201274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:09.200851    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5" event={"ID":"1ba3a57f-92e5-4de4-acc1-277d1574bffe","Type":"ContainerStarted","Data":"c4ffbf61bf00b2de6a98482351d4832e2d46822237b6b9291883ecee131c7f56"}
Sep 10 15:33:09.201274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:09.200896    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:09.219355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:09.219307    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5" podStartSLOduration=2.219293364 podStartE2EDuration="2.219293364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:09.217171823 +0000 UTC m=+1029.753127409" watchObservedRunningTime="2026-09-10 15:33:09.219293364 +0000 UTC m=+1029.755248926"
Sep 10 15:33:09.235610 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:09.235577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:10.677786 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:10.677751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:10.783418 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:10.783383    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:15.178650 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:15.178619    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:33:16.740360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:16.740322    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:33:20.207810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:20.207782    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:33:23.550424 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:23.550385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:24.162165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:24.162125    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:24.492426 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:24.492342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:26.183207 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:26.183176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:26.493780 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:26.493697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:26.808695 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:26.808609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:27.126005 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:27.125969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:28.111322 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:28.111290    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:30.398441 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:30.398409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:30.487232 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:30.487194    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:30.838020 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:30.837989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:31.156755 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:31.156673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:31.466760 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:31.466682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:31.695056 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:31.695016    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:31.793975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:31.793887    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:32.041475 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.041422    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:32.274424 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.274391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:32.365229 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.365193    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"]
Sep 10 15:33:32.368668 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.368653    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.371531 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.371511    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-d7a2b3\""
Sep 10 15:33:32.377708 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.377685    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"]
Sep 10 15:33:32.381127 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.381111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.383273 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.383254    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-d7a2b3-dockercfg-ss28m\""
Sep 10 15:33:32.387768 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.387748    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"]
Sep 10 15:33:32.392629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.392601    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"]
Sep 10 15:33:32.404381 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.404355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:32.410901 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.410861    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a76ea2-4e85-4017-bf3a-de8546983c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.411107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.411082    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kxfg\" (UniqueName: \"kubernetes.io/projected/42a76ea2-4e85-4017-bf3a-de8546983c90-kube-api-access-7kxfg\") pod \"payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.512167 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.512127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kxfg\" (UniqueName: \"kubernetes.io/projected/42a76ea2-4e85-4017-bf3a-de8546983c90-kube-api-access-7kxfg\") pod \"payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.512578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.512190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a76ea2-4e85-4017-bf3a-de8546983c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.512578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.512243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl5xm\" (UniqueName: \"kubernetes.io/projected/03154feb-1cfa-4764-9a67-a2ecc972cc00-kube-api-access-vl5xm\") pod \"payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.512578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.512306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03154feb-1cfa-4764-9a67-a2ecc972cc00-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.512887 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.512869    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a76ea2-4e85-4017-bf3a-de8546983c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.522002 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.521967    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kxfg\" (UniqueName: \"kubernetes.io/projected/42a76ea2-4e85-4017-bf3a-de8546983c90-kube-api-access-7kxfg\") pod \"payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.613127 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.613094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl5xm\" (UniqueName: \"kubernetes.io/projected/03154feb-1cfa-4764-9a67-a2ecc972cc00-kube-api-access-vl5xm\") pod \"payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.613328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.613182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03154feb-1cfa-4764-9a67-a2ecc972cc00-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.613926 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.613901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03154feb-1cfa-4764-9a67-a2ecc972cc00-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.621431 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.621409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl5xm\" (UniqueName: \"kubernetes.io/projected/03154feb-1cfa-4764-9a67-a2ecc972cc00-kube-api-access-vl5xm\") pod \"payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.679098 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.679048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:32.692172 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.692136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:32.955141 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.955115    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"]
Sep 10 15:33:32.957219 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:33:32.957192    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a76ea2_4e85_4017_bf3a_de8546983c90.slice/crio-81d2b2f8887559d1147d5a489ec9b3bb131a44a44e3a42033ce7613ae73388b8 WatchSource:0}: Error finding container 81d2b2f8887559d1147d5a489ec9b3bb131a44a44e3a42033ce7613ae73388b8: Status 404 returned error can't find the container with id 81d2b2f8887559d1147d5a489ec9b3bb131a44a44e3a42033ce7613ae73388b8
Sep 10 15:33:32.982505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:32.982482    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"]
Sep 10 15:33:33.001728 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:33:33.001703    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03154feb_1cfa_4764_9a67_a2ecc972cc00.slice/crio-c62a1ca70cbceb0cacee9c78da01ffb9261d04b8ad505e2e3a96e7f0411a2070 WatchSource:0}: Error finding container c62a1ca70cbceb0cacee9c78da01ffb9261d04b8ad505e2e3a96e7f0411a2070: Status 404 returned error can't find the container with id c62a1ca70cbceb0cacee9c78da01ffb9261d04b8ad505e2e3a96e7f0411a2070
Sep 10 15:33:33.323207 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.323172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d" event={"ID":"03154feb-1cfa-4764-9a67-a2ecc972cc00","Type":"ContainerStarted","Data":"0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486"}
Sep 10 15:33:33.323400 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.323227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d" event={"ID":"03154feb-1cfa-4764-9a67-a2ecc972cc00","Type":"ContainerStarted","Data":"c62a1ca70cbceb0cacee9c78da01ffb9261d04b8ad505e2e3a96e7f0411a2070"}
Sep 10 15:33:33.323400 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.323270    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:33.324762 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.324736    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74" event={"ID":"42a76ea2-4e85-4017-bf3a-de8546983c90","Type":"ContainerStarted","Data":"885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543"}
Sep 10 15:33:33.324881 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.324780    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74" event={"ID":"42a76ea2-4e85-4017-bf3a-de8546983c90","Type":"ContainerStarted","Data":"81d2b2f8887559d1147d5a489ec9b3bb131a44a44e3a42033ce7613ae73388b8"}
Sep 10 15:33:33.324881 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.324803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:33.342321 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.342271    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d" podStartSLOduration=1.342253981 podStartE2EDuration="1.342253981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:33.340970741 +0000 UTC m=+1053.876926317" watchObservedRunningTime="2026-09-10 15:33:33.342253981 +0000 UTC m=+1053.878209568"
Sep 10 15:33:33.359313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:33.359265    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74" podStartSLOduration=1.359249275 podStartE2EDuration="1.359249275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:33.357340083 +0000 UTC m=+1053.893295649" watchObservedRunningTime="2026-09-10 15:33:33.359249275 +0000 UTC m=+1053.895204836"
Sep 10 15:33:34.373599 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:34.373561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:34.464020 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:34.463991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:34.694508 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:34.694411    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:33:35.589731 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:35.589696    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:35.688072 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:35.688035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:35.876396 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:35.876310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:36.092076 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.092043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:36.192547 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.192444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:36.211702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.211669    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"]
Sep 10 15:33:36.215436 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.215420    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.218466 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.218429    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-fc3e54-dockercfg-46x6c\""
Sep 10 15:33:36.218466 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.218446    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-fc3e54\""
Sep 10 15:33:36.228034 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.228009    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"]
Sep 10 15:33:36.278718 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.278688    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"]
Sep 10 15:33:36.282138 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.282120    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.293366 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.293344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"]
Sep 10 15:33:36.352183 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.352151    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57260c99-a3a6-4389-a4f7-bf1dd669d77d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.352374 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.352219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69n7f\" (UniqueName: \"kubernetes.io/projected/d79cd1fc-5112-4e33-83a0-c46f038c485a-kube-api-access-69n7f\") pod \"payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.352374 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.352330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wpwf\" (UniqueName: \"kubernetes.io/projected/57260c99-a3a6-4389-a4f7-bf1dd669d77d-kube-api-access-5wpwf\") pod \"payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.352515 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.352394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79cd1fc-5112-4e33-83a0-c46f038c485a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.453798 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.453704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57260c99-a3a6-4389-a4f7-bf1dd669d77d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.453798 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.453773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69n7f\" (UniqueName: \"kubernetes.io/projected/d79cd1fc-5112-4e33-83a0-c46f038c485a-kube-api-access-69n7f\") pod \"payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.454036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.453818    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wpwf\" (UniqueName: \"kubernetes.io/projected/57260c99-a3a6-4389-a4f7-bf1dd669d77d-kube-api-access-5wpwf\") pod \"payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.454036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.453847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79cd1fc-5112-4e33-83a0-c46f038c485a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.454573 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.454548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57260c99-a3a6-4389-a4f7-bf1dd669d77d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.454667 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.454553    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79cd1fc-5112-4e33-83a0-c46f038c485a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.462544 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.462520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69n7f\" (UniqueName: \"kubernetes.io/projected/d79cd1fc-5112-4e33-83a0-c46f038c485a-kube-api-access-69n7f\") pod \"payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.462882 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.462863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wpwf\" (UniqueName: \"kubernetes.io/projected/57260c99-a3a6-4389-a4f7-bf1dd669d77d-kube-api-access-5wpwf\") pod \"payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.526517 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.526477    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:36.593305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.593267    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:36.800150 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.800124    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"]
Sep 10 15:33:36.801734 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:33:36.801702    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79cd1fc_5112_4e33_83a0_c46f038c485a.slice/crio-e78c79e48e25a45068d2a3cb18e0bb6273f88a511f2eba3cb89d5a47c4e25273 WatchSource:0}: Error finding container e78c79e48e25a45068d2a3cb18e0bb6273f88a511f2eba3cb89d5a47c4e25273: Status 404 returned error can't find the container with id e78c79e48e25a45068d2a3cb18e0bb6273f88a511f2eba3cb89d5a47c4e25273
Sep 10 15:33:36.868004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:36.867981    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"]
Sep 10 15:33:36.869049 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:33:36.869025    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57260c99_a3a6_4389_a4f7_bf1dd669d77d.slice/crio-c8c0cfdf0f9243486311846c9ef0efacb1c04528afd349948e0a912ff46c9359 WatchSource:0}: Error finding container c8c0cfdf0f9243486311846c9ef0efacb1c04528afd349948e0a912ff46c9359: Status 404 returned error can't find the container with id c8c0cfdf0f9243486311846c9ef0efacb1c04528afd349948e0a912ff46c9359
Sep 10 15:33:37.347818 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.347766    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh" event={"ID":"d79cd1fc-5112-4e33-83a0-c46f038c485a","Type":"ContainerStarted","Data":"5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800"}
Sep 10 15:33:37.348050 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.347839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh" event={"ID":"d79cd1fc-5112-4e33-83a0-c46f038c485a","Type":"ContainerStarted","Data":"e78c79e48e25a45068d2a3cb18e0bb6273f88a511f2eba3cb89d5a47c4e25273"}
Sep 10 15:33:37.348050 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.347868    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:37.349416 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.349390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j" event={"ID":"57260c99-a3a6-4389-a4f7-bf1dd669d77d","Type":"ContainerStarted","Data":"655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9"}
Sep 10 15:33:37.349557 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.349436    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j" event={"ID":"57260c99-a3a6-4389-a4f7-bf1dd669d77d","Type":"ContainerStarted","Data":"c8c0cfdf0f9243486311846c9ef0efacb1c04528afd349948e0a912ff46c9359"}
Sep 10 15:33:37.349557 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.349508    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:37.369563 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.369510    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh" podStartSLOduration=1.369493973 podStartE2EDuration="1.369493973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:37.368556168 +0000 UTC m=+1057.904511748" watchObservedRunningTime="2026-09-10 15:33:37.369493973 +0000 UTC m=+1057.905449535"
Sep 10 15:33:37.387467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:37.387398    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j" podStartSLOduration=1.387382115 podStartE2EDuration="1.387382115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:37.385939686 +0000 UTC m=+1057.921895277" watchObservedRunningTime="2026-09-10 15:33:37.387382115 +0000 UTC m=+1057.923337677"
Sep 10 15:33:44.181858 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:44.181820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:44.243715 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:44.243684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:44.331560 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:44.331530    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:33:44.331727 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:44.331610    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:33:46.172341 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:46.172307    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:33:48.357835 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:48.357803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:33:48.358236 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:48.357888    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:33:53.612969 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:53.612889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:55.305311 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:55.305280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:56.551266 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:56.551231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:59.297021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:33:59.296984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:04.928959 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:04.928924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:05.275839 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:05.275749    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:05.435967 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:05.435930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:08.649046 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:08.649000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:08.743318 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:08.743286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:08.839169 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:08.839134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:09.966906 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:09.966873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:10.090815 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:10.090775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:10.868062 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:10.868030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:10.975693 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:10.975652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:11.151167 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.151075    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"]
Sep 10 15:34:11.156227 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.156202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.166710 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.166686    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-26935da7\""
Sep 10 15:34:11.166841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.166791    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-26935da7-dockercfg-l6knj\""
Sep 10 15:34:11.187133 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.187098    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"]
Sep 10 15:34:11.293727 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.293690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48137e25-a585-4b08-85a5-d1a8f30cbcd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.293908 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.293792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxzx2\" (UniqueName: \"kubernetes.io/projected/48137e25-a585-4b08-85a5-d1a8f30cbcd0-kube-api-access-cxzx2\") pod \"payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.395417 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.395379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48137e25-a585-4b08-85a5-d1a8f30cbcd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.395603 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.395436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxzx2\" (UniqueName: \"kubernetes.io/projected/48137e25-a585-4b08-85a5-d1a8f30cbcd0-kube-api-access-cxzx2\") pod \"payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.396195 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.396176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48137e25-a585-4b08-85a5-d1a8f30cbcd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.418612 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.418541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxzx2\" (UniqueName: \"kubernetes.io/projected/48137e25-a585-4b08-85a5-d1a8f30cbcd0-kube-api-access-cxzx2\") pod \"payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.467305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.467263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:11.748950 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:11.748863    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"]
Sep 10 15:34:11.751917 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:34:11.751890    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48137e25_a585_4b08_85a5_d1a8f30cbcd0.slice/crio-e502e760fda9df6a44b2ddddbce1853105c51f7a6b436a90881d9601cf80d97d WatchSource:0}: Error finding container e502e760fda9df6a44b2ddddbce1853105c51f7a6b436a90881d9601cf80d97d: Status 404 returned error can't find the container with id e502e760fda9df6a44b2ddddbce1853105c51f7a6b436a90881d9601cf80d97d
Sep 10 15:34:12.545671 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:12.545628    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c" event={"ID":"48137e25-a585-4b08-85a5-d1a8f30cbcd0","Type":"ContainerStarted","Data":"57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea"}
Sep 10 15:34:12.546069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:12.545690    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c" event={"ID":"48137e25-a585-4b08-85a5-d1a8f30cbcd0","Type":"ContainerStarted","Data":"e502e760fda9df6a44b2ddddbce1853105c51f7a6b436a90881d9601cf80d97d"}
Sep 10 15:34:12.546069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:12.545738    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:12.568697 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:12.568625    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c" podStartSLOduration=1.568608472 podStartE2EDuration="1.568608472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:34:12.568277931 +0000 UTC m=+1093.104233519" watchObservedRunningTime="2026-09-10 15:34:12.568608472 +0000 UTC m=+1093.104564037"
Sep 10 15:34:17.641402 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:17.641364    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:34:23.554071 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:23.554035    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:34:33.572518 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:33.572475    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:43.509211 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:43.509164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:48.566519 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:34:48.566476    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:35:13.967371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:13.967324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:21.971289 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.971255    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"]
Sep 10 15:35:21.975265 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.975246    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:21.978865 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.978835    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"default-dockercfg-wvljq\""
Sep 10 15:35:21.978865 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.978836    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"auth-test-model-fc3e54-kserve-self-signed-certs\""
Sep 10 15:35:21.979060 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.978838    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"kube-root-ca.crt\""
Sep 10 15:35:21.979060 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.978838    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"openshift-service-ca.crt\""
Sep 10 15:35:21.984775 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.984745    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"]
Sep 10 15:35:21.993931 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.993898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:21.996097 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.996073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-model-cache\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:21.996219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.996102    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s585z\" (UniqueName: \"kubernetes.io/projected/52c89025-0d6f-4af4-84d4-d9fc6d89412e-kube-api-access-s585z\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:21.996219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.996150    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-home\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:21.996219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.996198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-dshm\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:21.996391 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.996243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tmp-dir\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:21.996391 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:21.996283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tls-certs\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097364 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tmp-dir\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097364 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tls-certs\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-model-cache\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s585z\" (UniqueName: \"kubernetes.io/projected/52c89025-0d6f-4af4-84d4-d9fc6d89412e-kube-api-access-s585z\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-home\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097645 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-dshm\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097877 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tmp-dir\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097877 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097827    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-model-cache\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.097959 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.097880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-home\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.099716 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.099698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-dshm\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.099899 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.099883    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tls-certs\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.110834 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.110803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s585z\" (UniqueName: \"kubernetes.io/projected/52c89025-0d6f-4af4-84d4-d9fc6d89412e-kube-api-access-s585z\") pod \"auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.285911 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.285820    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:22.559445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.559415    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"]
Sep 10 15:35:22.561033 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:35:22.561004    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52c89025_0d6f_4af4_84d4_d9fc6d89412e.slice/crio-39cb68a978344d929b30697bcb0f1764bb18d1007a55cb32a41cd34c072f25e3 WatchSource:0}: Error finding container 39cb68a978344d929b30697bcb0f1764bb18d1007a55cb32a41cd34c072f25e3: Status 404 returned error can't find the container with id 39cb68a978344d929b30697bcb0f1764bb18d1007a55cb32a41cd34c072f25e3
Sep 10 15:35:22.933780 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.933743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v" event={"ID":"52c89025-0d6f-4af4-84d4-d9fc6d89412e","Type":"ContainerStarted","Data":"8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081"}
Sep 10 15:35:22.933780 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.933784    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v" event={"ID":"52c89025-0d6f-4af4-84d4-d9fc6d89412e","Type":"ContainerStarted","Data":"39cb68a978344d929b30697bcb0f1764bb18d1007a55cb32a41cd34c072f25e3"}
Sep 10 15:35:22.955315 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:22.955259    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v" podStartSLOduration=1.9552392429999998 podStartE2EDuration="1.955239243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:35:22.954199145 +0000 UTC m=+1163.490154719" watchObservedRunningTime="2026-09-10 15:35:22.955239243 +0000 UTC m=+1163.491195139"
Sep 10 15:35:26.046678 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:26.046638    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:35:32.286874 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:32.286827    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:32.287307 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:32.287104    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:32.299189 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:32.299164    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:32.997421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:32.997389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:35:55.744659 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:55.744621    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:35:59.167816 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:59.167763    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:59.968859 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:59.968832    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:35:59.970729 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:35:59.970705    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:36:13.330361 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:13.330322    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:36:14.560186 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.560147    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:14.590165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.590129    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"]
Sep 10 15:36:14.590434 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.590404    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c" podUID="48137e25-a585-4b08-85a5-d1a8f30cbcd0" containerName="payload-processing" containerID="cri-o://57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea" gracePeriod=30
Sep 10 15:36:14.633759 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.633727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:14.733876 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.733828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:14.801349 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.801313    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:14.842882 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.842855    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:36:14.940533 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.940502    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48137e25-a585-4b08-85a5-d1a8f30cbcd0-plugins-config-volume\") pod \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") "
Sep 10 15:36:14.940719 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.940575    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxzx2\" (UniqueName: \"kubernetes.io/projected/48137e25-a585-4b08-85a5-d1a8f30cbcd0-kube-api-access-cxzx2\") pod \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\" (UID: \"48137e25-a585-4b08-85a5-d1a8f30cbcd0\") "
Sep 10 15:36:14.940911 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.940887    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48137e25-a585-4b08-85a5-d1a8f30cbcd0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "48137e25-a585-4b08-85a5-d1a8f30cbcd0" (UID: "48137e25-a585-4b08-85a5-d1a8f30cbcd0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:36:14.942634 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:14.942612    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48137e25-a585-4b08-85a5-d1a8f30cbcd0-kube-api-access-cxzx2" (OuterVolumeSpecName: "kube-api-access-cxzx2") pod "48137e25-a585-4b08-85a5-d1a8f30cbcd0" (UID: "48137e25-a585-4b08-85a5-d1a8f30cbcd0"). InnerVolumeSpecName "kube-api-access-cxzx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:36:15.041596 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.041553    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48137e25-a585-4b08-85a5-d1a8f30cbcd0-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:36:15.041596 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.041587    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxzx2\" (UniqueName: \"kubernetes.io/projected/48137e25-a585-4b08-85a5-d1a8f30cbcd0-kube-api-access-cxzx2\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:36:15.214749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.214653    2562 generic.go:356] "Generic (PLEG): container finished" podID="48137e25-a585-4b08-85a5-d1a8f30cbcd0" containerID="57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea" exitCode=0
Sep 10 15:36:15.214749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.214697    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"
Sep 10 15:36:15.214749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.214739    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c" event={"ID":"48137e25-a585-4b08-85a5-d1a8f30cbcd0","Type":"ContainerDied","Data":"57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea"}
Sep 10 15:36:15.214982 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.214793    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c" event={"ID":"48137e25-a585-4b08-85a5-d1a8f30cbcd0","Type":"ContainerDied","Data":"e502e760fda9df6a44b2ddddbce1853105c51f7a6b436a90881d9601cf80d97d"}
Sep 10 15:36:15.214982 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.214832    2562 scope.go:117] "RemoveContainer" containerID="57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea"
Sep 10 15:36:15.227301 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.227277    2562 scope.go:117] "RemoveContainer" containerID="57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea"
Sep 10 15:36:15.227763 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:36:15.227741    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea\": container with ID starting with 57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea not found: ID does not exist" containerID="57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea"
Sep 10 15:36:15.227850 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.227776    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea"} err="failed to get container status \"57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea\": rpc error: code = NotFound desc = could not find container \"57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea\": container with ID starting with 57f5062d90058bba005f7960ed9703fa89f03c7a6642f95eaed961205da8beea not found: ID does not exist"
Sep 10 15:36:15.256287 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.256248    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"]
Sep 10 15:36:15.270072 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.270031    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-26935da7-5465454fdf-j4r7c"]
Sep 10 15:36:15.956408 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:15.956374    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48137e25-a585-4b08-85a5-d1a8f30cbcd0" path="/var/lib/kubelet/pods/48137e25-a585-4b08-85a5-d1a8f30cbcd0/volumes"
Sep 10 15:36:17.789270 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:17.789229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:22.872256 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:22.872214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:29.441827 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:29.441786    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:36:44.531938 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:44.531903    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:36:59.107931 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:59.107773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:59.534658 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:36:59.534568    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:37:00.029801 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:00.029769    2562 scope.go:117] "RemoveContainer" containerID="3bfd45e2f03fe00168bb764f1c4face71a286e009f0c3a7101a74bd69ca795cd"
Sep 10 15:37:00.038682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:00.038654    2562 scope.go:117] "RemoveContainer" containerID="02cb060cefe4308fe0f40d9670074d3c28a35335ea34a2d3fc0e7d7df9093ed9"
Sep 10 15:37:00.156513 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:00.156476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:06.251557 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:06.251520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:06.358964 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:06.358919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:06.669763 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:06.669720    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:07.867760 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:07.867717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:07.958231 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:07.958197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:08.057154 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:08.057110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:08.706786 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:08.706750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:08.805509 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:08.805410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:10.215467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:10.215418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:16.037624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:16.037583    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:37:16.783610 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:16.783570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:18.357857 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:18.357823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:23.318233 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:23.318191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:23.418285 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:23.418247    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:23.523110 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:23.523068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:31.937166 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:31.937129    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:37:33.991432 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:33.991390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:34.201115 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:34.201076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:48.532604 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:48.532568    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:37:54.514697 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:54.514655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:55.091411 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:55.091369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:55.529716 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:55.529624    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"]
Sep 10 15:37:55.549389 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:37:55.549356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:00.030626 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:00.030585    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:38:01.578722 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:01.578680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:01.643374 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:01.643341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:01.887942 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:01.887847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:02.334297 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:02.334262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:02.473503 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:02.473469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:05.152311 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:05.152276    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:05.279635 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:05.279600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:06.362085 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:06.362042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:06.538486 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:06.538431    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:06.671343 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:06.671226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:07.089095 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.089061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:07.179145 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.179105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:07.299794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.299755    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"]
Sep 10 15:38:07.300336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.300317    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48137e25-a585-4b08-85a5-d1a8f30cbcd0" containerName="payload-processing"
Sep 10 15:38:07.300391 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.300337    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="48137e25-a585-4b08-85a5-d1a8f30cbcd0" containerName="payload-processing"
Sep 10 15:38:07.300427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.300420    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="48137e25-a585-4b08-85a5-d1a8f30cbcd0" containerName="payload-processing"
Sep 10 15:38:07.303818 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.303803    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.312903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.312872    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-dd499d\""
Sep 10 15:38:07.312903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.312887    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-dd499d-dockercfg-2rnsz\""
Sep 10 15:38:07.325187 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.325153    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"]
Sep 10 15:38:07.417444 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.417337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk6p9\" (UniqueName: \"kubernetes.io/projected/b5e03482-6c9d-4ff4-9357-3aca0f28b803-kube-api-access-xk6p9\") pod \"payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.417865 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.417521    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e03482-6c9d-4ff4-9357-3aca0f28b803-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.518301 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.518257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e03482-6c9d-4ff4-9357-3aca0f28b803-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.518526 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.518363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk6p9\" (UniqueName: \"kubernetes.io/projected/b5e03482-6c9d-4ff4-9357-3aca0f28b803-kube-api-access-xk6p9\") pod \"payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.519007 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.518983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e03482-6c9d-4ff4-9357-3aca0f28b803-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.532700 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.532674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk6p9\" (UniqueName: \"kubernetes.io/projected/b5e03482-6c9d-4ff4-9357-3aca0f28b803-kube-api-access-xk6p9\") pod \"payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.620558 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.620513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:07.909091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:07.909058    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"]
Sep 10 15:38:07.910721 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:38:07.910680    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e03482_6c9d_4ff4_9357_3aca0f28b803.slice/crio-4c4f3c6b84e4be9e3307cf12ba21281d503c127b83ea4ab30898beee0e55e080 WatchSource:0}: Error finding container 4c4f3c6b84e4be9e3307cf12ba21281d503c127b83ea4ab30898beee0e55e080: Status 404 returned error can't find the container with id 4c4f3c6b84e4be9e3307cf12ba21281d503c127b83ea4ab30898beee0e55e080
Sep 10 15:38:08.827045 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:08.827012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb" event={"ID":"b5e03482-6c9d-4ff4-9357-3aca0f28b803","Type":"ContainerStarted","Data":"59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7"}
Sep 10 15:38:08.827430 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:08.827065    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb" event={"ID":"b5e03482-6c9d-4ff4-9357-3aca0f28b803","Type":"ContainerStarted","Data":"4c4f3c6b84e4be9e3307cf12ba21281d503c127b83ea4ab30898beee0e55e080"}
Sep 10 15:38:08.827430 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:08.827105    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:08.858877 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:08.858816    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb" podStartSLOduration=1.8587995309999998 podStartE2EDuration="1.858799531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:38:08.856444123 +0000 UTC m=+1329.392399712" watchObservedRunningTime="2026-09-10 15:38:08.858799531 +0000 UTC m=+1329.394755093"
Sep 10 15:38:10.550796 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.550732    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v" podUID="52c89025-0d6f-4af4-84d4-d9fc6d89412e" containerName="main" containerID="cri-o://8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081" gracePeriod=45
Sep 10 15:38:10.806578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.806510    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:38:10.840472 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.840413    2562 generic.go:356] "Generic (PLEG): container finished" podID="52c89025-0d6f-4af4-84d4-d9fc6d89412e" containerID="8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081" exitCode=0
Sep 10 15:38:10.840656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.840487    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"
Sep 10 15:38:10.840656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.840506    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v" event={"ID":"52c89025-0d6f-4af4-84d4-d9fc6d89412e","Type":"ContainerDied","Data":"8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081"}
Sep 10 15:38:10.840656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.840559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v" event={"ID":"52c89025-0d6f-4af4-84d4-d9fc6d89412e","Type":"ContainerDied","Data":"39cb68a978344d929b30697bcb0f1764bb18d1007a55cb32a41cd34c072f25e3"}
Sep 10 15:38:10.840656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.840584    2562 scope.go:117] "RemoveContainer" containerID="8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081"
Sep 10 15:38:10.854103 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.854065    2562 scope.go:117] "RemoveContainer" containerID="8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081"
Sep 10 15:38:10.854889 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:38:10.854855    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081\": container with ID starting with 8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081 not found: ID does not exist" containerID="8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081"
Sep 10 15:38:10.855037 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.854901    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081"} err="failed to get container status \"8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081\": rpc error: code = NotFound desc = could not find container \"8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081\": container with ID starting with 8389debb11ce8b1c32ea424e0f65e64b590842e86f40a1749527eeecc8bbd081 not found: ID does not exist"
Sep 10 15:38:10.955548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955517    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s585z\" (UniqueName: \"kubernetes.io/projected/52c89025-0d6f-4af4-84d4-d9fc6d89412e-kube-api-access-s585z\") pod \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") "
Sep 10 15:38:10.955548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955553    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-dshm\") pod \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") "
Sep 10 15:38:10.955800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955600    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-model-cache\") pod \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") "
Sep 10 15:38:10.955800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955646    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-home\") pod \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") "
Sep 10 15:38:10.955800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955685    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tls-certs\") pod \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") "
Sep 10 15:38:10.955800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955702    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tmp-dir\") pod \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\" (UID: \"52c89025-0d6f-4af4-84d4-d9fc6d89412e\") "
Sep 10 15:38:10.956006 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955899    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-model-cache" (OuterVolumeSpecName: "model-cache") pod "52c89025-0d6f-4af4-84d4-d9fc6d89412e" (UID: "52c89025-0d6f-4af4-84d4-d9fc6d89412e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:38:10.956006 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.955942    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-home" (OuterVolumeSpecName: "home") pod "52c89025-0d6f-4af4-84d4-d9fc6d89412e" (UID: "52c89025-0d6f-4af4-84d4-d9fc6d89412e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:38:10.956111 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.956033    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52c89025-0d6f-4af4-84d4-d9fc6d89412e" (UID: "52c89025-0d6f-4af4-84d4-d9fc6d89412e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:38:10.956111 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.956097    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:38:10.956198 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.956115    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:38:10.958045 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.958012    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52c89025-0d6f-4af4-84d4-d9fc6d89412e" (UID: "52c89025-0d6f-4af4-84d4-d9fc6d89412e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:38:10.958045 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.958012    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-dshm" (OuterVolumeSpecName: "dshm") pod "52c89025-0d6f-4af4-84d4-d9fc6d89412e" (UID: "52c89025-0d6f-4af4-84d4-d9fc6d89412e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:38:10.958190 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:10.958084    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c89025-0d6f-4af4-84d4-d9fc6d89412e-kube-api-access-s585z" (OuterVolumeSpecName: "kube-api-access-s585z") pod "52c89025-0d6f-4af4-84d4-d9fc6d89412e" (UID: "52c89025-0d6f-4af4-84d4-d9fc6d89412e"). InnerVolumeSpecName "kube-api-access-s585z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:38:11.057647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.057551    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:38:11.057647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.057586    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:38:11.057647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.057595    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s585z\" (UniqueName: \"kubernetes.io/projected/52c89025-0d6f-4af4-84d4-d9fc6d89412e-kube-api-access-s585z\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:38:11.057647 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.057604    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c89025-0d6f-4af4-84d4-d9fc6d89412e-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:38:11.163410 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.163374    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"]
Sep 10 15:38:11.169069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.169041    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/auth-test-model-fc3e54-kserve-8664bfbf79-xhg6v"]
Sep 10 15:38:11.510527 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.510445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:11.956106 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:11.956061    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c89025-0d6f-4af4-84d4-d9fc6d89412e" path="/var/lib/kubelet/pods/52c89025-0d6f-4af4-84d4-d9fc6d89412e/volumes"
Sep 10 15:38:12.008192 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:12.008157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:13.737036 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:13.736999    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:38:19.330820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.330779    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"]
Sep 10 15:38:19.331445 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.331424    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c89025-0d6f-4af4-84d4-d9fc6d89412e" containerName="main"
Sep 10 15:38:19.331578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.331474    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c89025-0d6f-4af4-84d4-d9fc6d89412e" containerName="main"
Sep 10 15:38:19.331638 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.331590    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c89025-0d6f-4af4-84d4-d9fc6d89412e" containerName="main"
Sep 10 15:38:19.336038 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.336017    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.338983 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.338960    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-62d2f7\"/\"default-dockercfg-mfpk6\""
Sep 10 15:38:19.339290 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.339268    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-62d2f7\"/\"openshift-service-ca.crt\""
Sep 10 15:38:19.339489 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.339471    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-62d2f7\"/\"kube-root-ca.crt\""
Sep 10 15:38:19.339686 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.339666    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-62d2f7\"/\"sub-test-model-62d2f7-kserve-self-signed-certs\""
Sep 10 15:38:19.341095 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.341064    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"]
Sep 10 15:38:19.363168 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.363140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:19.441125 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.441085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4xgc\" (UniqueName: \"kubernetes.io/projected/d5f3202b-89fd-41e7-8661-9abb4c419df5-kube-api-access-k4xgc\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.441312 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.441137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-tmp-dir\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.441312 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.441160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-home\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.441312 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.441238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-dshm\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.441312 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.441276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f3202b-89fd-41e7-8661-9abb4c419df5-tls-certs\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.441506 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.441339    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-model-cache\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.542560 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.542511    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-dshm\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.542560 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.542570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f3202b-89fd-41e7-8661-9abb4c419df5-tls-certs\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.542831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.542607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-model-cache\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.542831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.542673    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4xgc\" (UniqueName: \"kubernetes.io/projected/d5f3202b-89fd-41e7-8661-9abb4c419df5-kube-api-access-k4xgc\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.542831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.542707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-tmp-dir\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.542831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.542738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-home\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.543082 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.543049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-model-cache\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.543146 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.543055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-home\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.543146 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.543114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-tmp-dir\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.544817 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.544784    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-dshm\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.545059 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.545044    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f3202b-89fd-41e7-8661-9abb4c419df5-tls-certs\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.553468 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.553412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4xgc\" (UniqueName: \"kubernetes.io/projected/d5f3202b-89fd-41e7-8661-9abb4c419df5-kube-api-access-k4xgc\") pod \"sub-test-model-62d2f7-kserve-78d978ff65-t27hz\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") " pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.647785 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.647686    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:19.834487 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.834424    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:38:19.923551 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:19.923520    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"]
Sep 10 15:38:19.924997 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:38:19.924966    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5f3202b_89fd_41e7_8661_9abb4c419df5.slice/crio-b94e7152d18bc32dd1de5d5024b84aab6c14428c1ed7e543b9a047b64806f64f WatchSource:0}: Error finding container b94e7152d18bc32dd1de5d5024b84aab6c14428c1ed7e543b9a047b64806f64f: Status 404 returned error can't find the container with id b94e7152d18bc32dd1de5d5024b84aab6c14428c1ed7e543b9a047b64806f64f
Sep 10 15:38:20.904694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:20.904650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz" event={"ID":"d5f3202b-89fd-41e7-8661-9abb4c419df5","Type":"ContainerStarted","Data":"91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9"}
Sep 10 15:38:20.904694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:20.904700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz" event={"ID":"d5f3202b-89fd-41e7-8661-9abb4c419df5","Type":"ContainerStarted","Data":"b94e7152d18bc32dd1de5d5024b84aab6c14428c1ed7e543b9a047b64806f64f"}
Sep 10 15:38:20.923717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:20.923661    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz" podStartSLOduration=1.9236446680000001 podStartE2EDuration="1.923644668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:38:20.922365339 +0000 UTC m=+1341.458320930" watchObservedRunningTime="2026-09-10 15:38:20.923644668 +0000 UTC m=+1341.459600229"
Sep 10 15:38:24.477009 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:24.476893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:25.941107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:25.941074    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:38:28.861095 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:28.861056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:29.648065 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:29.648028    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:29.648260 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:29.648086    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:29.660778 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:29.660749    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:29.730080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:29.730044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:29.964879 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:29.964784    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:38:35.181152 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:35.181104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:35.653991 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:35.653942    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:36.088392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:36.088354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:36.845570 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:36.845533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:38.555691 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:38.555655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:39.320984 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:39.320953    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:39.421467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:39.421404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:40.505426 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:40.505392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:40.602696 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:40.602661    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:40.805249 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:40.805160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:41.002091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.002055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:41.100924 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.100889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:41.128349 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.128307    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"]
Sep 10 15:38:41.132135 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.132111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.134694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.134668    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-bbc967\""
Sep 10 15:38:41.134831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.134693    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-bbc967-dockercfg-bmd29\""
Sep 10 15:38:41.142738 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.142713    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"]
Sep 10 15:38:41.260181 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.260140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frlw4\" (UniqueName: \"kubernetes.io/projected/15fd3bc5-1297-418a-99e4-189b8e7aad62-kube-api-access-frlw4\") pod \"payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.260417 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.260194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fd3bc5-1297-418a-99e4-189b8e7aad62-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.361289 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.361195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fd3bc5-1297-418a-99e4-189b8e7aad62-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.361443 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.361329    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frlw4\" (UniqueName: \"kubernetes.io/projected/15fd3bc5-1297-418a-99e4-189b8e7aad62-kube-api-access-frlw4\") pod \"payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.361960 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.361939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fd3bc5-1297-418a-99e4-189b8e7aad62-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.370992 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.370964    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frlw4\" (UniqueName: \"kubernetes.io/projected/15fd3bc5-1297-418a-99e4-189b8e7aad62-kube-api-access-frlw4\") pod \"payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.443965 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.443909    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:41.722208 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:41.722182    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"]
Sep 10 15:38:41.723600 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:38:41.723570    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fd3bc5_1297_418a_99e4_189b8e7aad62.slice/crio-71ff2ef82908e6a8ef6e486dea9221b7237ee8bf835baee4aeefccf72231c4d3 WatchSource:0}: Error finding container 71ff2ef82908e6a8ef6e486dea9221b7237ee8bf835baee4aeefccf72231c4d3: Status 404 returned error can't find the container with id 71ff2ef82908e6a8ef6e486dea9221b7237ee8bf835baee4aeefccf72231c4d3
Sep 10 15:38:42.015170 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:42.015070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w" event={"ID":"15fd3bc5-1297-418a-99e4-189b8e7aad62","Type":"ContainerStarted","Data":"95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f"}
Sep 10 15:38:42.015170 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:42.015135    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w" event={"ID":"15fd3bc5-1297-418a-99e4-189b8e7aad62","Type":"ContainerStarted","Data":"71ff2ef82908e6a8ef6e486dea9221b7237ee8bf835baee4aeefccf72231c4d3"}
Sep 10 15:38:42.015371 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:42.015254    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:42.035271 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:42.035218    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w" podStartSLOduration=1.035201692 podStartE2EDuration="1.035201692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:38:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:38:42.032910103 +0000 UTC m=+1362.568865688" watchObservedRunningTime="2026-09-10 15:38:42.035201692 +0000 UTC m=+1362.571157253"
Sep 10 15:38:45.135638 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:45.135600    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:38:50.228258 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.228222    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"]
Sep 10 15:38:50.232346 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.232320    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.236115 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.236087    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"openshift-service-ca.crt\""
Sep 10 15:38:50.236264 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.236229    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"default-dockercfg-wvljq\""
Sep 10 15:38:50.236512 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.236489    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"kube-root-ca.crt\""
Sep 10 15:38:50.236627 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.236569    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"sub-test-model-fc3e54-kserve-self-signed-certs\""
Sep 10 15:38:50.243407 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.243381    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"]
Sep 10 15:38:50.261427 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.261371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:50.356345 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.356306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-model-cache\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.356547 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.356358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tls-certs\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.356547 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.356431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-home\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.356547 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.356532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-dshm\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.356663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.356579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tmp-dir\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.356663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.356598    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pgdv\" (UniqueName: \"kubernetes.io/projected/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-kube-api-access-2pgdv\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457228 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-model-cache\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457412 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tls-certs\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457412 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-home\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457412 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-dshm\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457579 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tmp-dir\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457579 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pgdv\" (UniqueName: \"kubernetes.io/projected/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-kube-api-access-2pgdv\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457691 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-model-cache\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457734 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-home\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.457796 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.457776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tmp-dir\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.459636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.459618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-dshm\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.459993 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.459977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tls-certs\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.466948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.466929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pgdv\" (UniqueName: \"kubernetes.io/projected/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-kube-api-access-2pgdv\") pod \"sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.546006 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.545921    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:38:50.819423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:50.819396    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"]
Sep 10 15:38:50.821095 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:38:50.821067    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc0f7596_2d6c_4ff9_8d65_54c823b6c7db.slice/crio-8338fab3267352688de808f26b675a5d300b5e0465b2075f9d99fa98f4d8ffd9 WatchSource:0}: Error finding container 8338fab3267352688de808f26b675a5d300b5e0465b2075f9d99fa98f4d8ffd9: Status 404 returned error can't find the container with id 8338fab3267352688de808f26b675a5d300b5e0465b2075f9d99fa98f4d8ffd9
Sep 10 15:38:51.064896 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:51.064856    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m" event={"ID":"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db","Type":"ContainerStarted","Data":"3552ebe131d39f43a2ac3855dcb2ee696b72ffcbff7dd01a4721dc7bd84aaa5a"}
Sep 10 15:38:51.064896 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:51.064901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m" event={"ID":"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db","Type":"ContainerStarted","Data":"8338fab3267352688de808f26b675a5d300b5e0465b2075f9d99fa98f4d8ffd9"}
Sep 10 15:38:51.085040 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:51.084921    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m" podStartSLOduration=1.084903161 podStartE2EDuration="1.084903161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:38:51.08293631 +0000 UTC m=+1371.618891899" watchObservedRunningTime="2026-09-10 15:38:51.084903161 +0000 UTC m=+1371.620858723"
Sep 10 15:38:53.022579 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:53.022543    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:38:54.424467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:54.424418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:55.191359 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:55.191318    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:57.269440 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:38:57.269379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:00.076608 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:00.076576    2562 scope.go:117] "RemoveContainer" containerID="ed3758fe90880afa34dc24d7aa9d25e92f2ca7341d15b423638eb2554102dd45"
Sep 10 15:39:00.546656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:00.546542    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:39:00.546656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:00.546596    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:39:00.559888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:00.559855    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:39:01.129125 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:01.129099    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:39:01.569081 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:01.569051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:04.289492 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:04.289419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:04.537184 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:04.537144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:07.182001 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.181959    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"]
Sep 10 15:39:07.189416 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.189393    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"
Sep 10 15:39:07.194269 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.194234    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:39:07.195024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.195000    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:39:07.195136 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.195095    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"]
Sep 10 15:39:07.223884 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.223837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e64bcf-d938-415a-b32a-48936eee09ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2\" (UID: \"27e64bcf-d938-415a-b32a-48936eee09ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"
Sep 10 15:39:07.325390 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.325346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e64bcf-d938-415a-b32a-48936eee09ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2\" (UID: \"27e64bcf-d938-415a-b32a-48936eee09ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"
Sep 10 15:39:07.326004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.325981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e64bcf-d938-415a-b32a-48936eee09ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2\" (UID: \"27e64bcf-d938-415a-b32a-48936eee09ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"
Sep 10 15:39:07.490163 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.490075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:07.502010 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.501981    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"
Sep 10 15:39:07.639839 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.639792    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:39:07.788185 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:39:07.788158    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e64bcf_d938_415a_b32a_48936eee09ba.slice/crio-2d92988dac109cd4ebe86620d2d371c397dc247d3831198e0bf712290b525f03 WatchSource:0}: Error finding container 2d92988dac109cd4ebe86620d2d371c397dc247d3831198e0bf712290b525f03: Status 404 returned error can't find the container with id 2d92988dac109cd4ebe86620d2d371c397dc247d3831198e0bf712290b525f03
Sep 10 15:39:07.790500 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:07.790466    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"]
Sep 10 15:39:08.164936 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:08.164901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2" event={"ID":"27e64bcf-d938-415a-b32a-48936eee09ba","Type":"ContainerStarted","Data":"e29e737de0ed9bc6b94d6ea0f2226b48f51be20ea0523ad447ca0eadc7dd68ba"}
Sep 10 15:39:08.165122 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:08.164977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2" event={"ID":"27e64bcf-d938-415a-b32a-48936eee09ba","Type":"ContainerStarted","Data":"2d92988dac109cd4ebe86620d2d371c397dc247d3831198e0bf712290b525f03"}
Sep 10 15:39:08.183336 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:08.183280    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2" podStartSLOduration=1.183264183 podStartE2EDuration="1.183264183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:39:08.180508776 +0000 UTC m=+1388.716464365" watchObservedRunningTime="2026-09-10 15:39:08.183264183 +0000 UTC m=+1388.719219745"
Sep 10 15:39:09.171770 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:09.171729    2562 generic.go:356] "Generic (PLEG): container finished" podID="27e64bcf-d938-415a-b32a-48936eee09ba" containerID="e29e737de0ed9bc6b94d6ea0f2226b48f51be20ea0523ad447ca0eadc7dd68ba" exitCode=0
Sep 10 15:39:09.171961 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:09.171812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2" event={"ID":"27e64bcf-d938-415a-b32a-48936eee09ba","Type":"ContainerDied","Data":"e29e737de0ed9bc6b94d6ea0f2226b48f51be20ea0523ad447ca0eadc7dd68ba"}
Sep 10 15:39:10.315384 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:10.315359    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"
Sep 10 15:39:10.356398 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:10.356367    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e64bcf-d938-415a-b32a-48936eee09ba-maas-api-service-ca\") pod \"27e64bcf-d938-415a-b32a-48936eee09ba\" (UID: \"27e64bcf-d938-415a-b32a-48936eee09ba\") "
Sep 10 15:39:10.356764 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:10.356738    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e64bcf-d938-415a-b32a-48936eee09ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27e64bcf-d938-415a-b32a-48936eee09ba" (UID: "27e64bcf-d938-415a-b32a-48936eee09ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:39:10.457490 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:10.457378    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e64bcf-d938-415a-b32a-48936eee09ba-maas-api-service-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:11.182929 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:11.182892    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"
Sep 10 15:39:11.182929 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:11.182921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2" event={"ID":"27e64bcf-d938-415a-b32a-48936eee09ba","Type":"ContainerDied","Data":"2d92988dac109cd4ebe86620d2d371c397dc247d3831198e0bf712290b525f03"}
Sep 10 15:39:11.183137 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:11.182967    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d92988dac109cd4ebe86620d2d371c397dc247d3831198e0bf712290b525f03"
Sep 10 15:39:22.783379 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.783317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:22.819837 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.819796    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"]
Sep 10 15:39:22.820079 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.820052    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb" podUID="b5e03482-6c9d-4ff4-9357-3aca0f28b803" containerName="payload-pre-processing" containerID="cri-o://59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7" gracePeriod=30
Sep 10 15:39:22.850247 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.850217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:22.925819 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.925787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:22.983793 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.983755    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"]
Sep 10 15:39:22.987355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.987320    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dd499d-vqfd2"]
Sep 10 15:39:22.992661 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:22.992217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:23.077623 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.077578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:23.084691 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.084664    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:39:23.186234 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.186197    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk6p9\" (UniqueName: \"kubernetes.io/projected/b5e03482-6c9d-4ff4-9357-3aca0f28b803-kube-api-access-xk6p9\") pod \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") "
Sep 10 15:39:23.186430 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.186250    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e03482-6c9d-4ff4-9357-3aca0f28b803-plugins-config-volume\") pod \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\" (UID: \"b5e03482-6c9d-4ff4-9357-3aca0f28b803\") "
Sep 10 15:39:23.186664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.186641    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5e03482-6c9d-4ff4-9357-3aca0f28b803-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5e03482-6c9d-4ff4-9357-3aca0f28b803" (UID: "b5e03482-6c9d-4ff4-9357-3aca0f28b803"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:39:23.188290 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.188268    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5e03482-6c9d-4ff4-9357-3aca0f28b803-kube-api-access-xk6p9" (OuterVolumeSpecName: "kube-api-access-xk6p9") pod "b5e03482-6c9d-4ff4-9357-3aca0f28b803" (UID: "b5e03482-6c9d-4ff4-9357-3aca0f28b803"). InnerVolumeSpecName "kube-api-access-xk6p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:39:23.246244 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.246212    2562 generic.go:356] "Generic (PLEG): container finished" podID="b5e03482-6c9d-4ff4-9357-3aca0f28b803" containerID="59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7" exitCode=0
Sep 10 15:39:23.246407 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.246255    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"
Sep 10 15:39:23.246407 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.246304    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb" event={"ID":"b5e03482-6c9d-4ff4-9357-3aca0f28b803","Type":"ContainerDied","Data":"59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7"}
Sep 10 15:39:23.246407 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.246353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb" event={"ID":"b5e03482-6c9d-4ff4-9357-3aca0f28b803","Type":"ContainerDied","Data":"4c4f3c6b84e4be9e3307cf12ba21281d503c127b83ea4ab30898beee0e55e080"}
Sep 10 15:39:23.246407 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.246376    2562 scope.go:117] "RemoveContainer" containerID="59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7"
Sep 10 15:39:23.256402 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.256385    2562 scope.go:117] "RemoveContainer" containerID="59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7"
Sep 10 15:39:23.256738 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:39:23.256719    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7\": container with ID starting with 59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7 not found: ID does not exist" containerID="59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7"
Sep 10 15:39:23.256816 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.256747    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7"} err="failed to get container status \"59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7\": rpc error: code = NotFound desc = could not find container \"59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7\": container with ID starting with 59a774fb33d1e6634b30537c4aecc3b82e9189ce596164887871543f11227ba7 not found: ID does not exist"
Sep 10 15:39:23.269276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.269246    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"]
Sep 10 15:39:23.273678 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.273653    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dd499d-85cf47bf54-849tb"]
Sep 10 15:39:23.287628 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.287605    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk6p9\" (UniqueName: \"kubernetes.io/projected/b5e03482-6c9d-4ff4-9357-3aca0f28b803-kube-api-access-xk6p9\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:23.287628 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.287627    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e03482-6c9d-4ff4-9357-3aca0f28b803-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:23.329069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.329043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:23.956439 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.956403    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27e64bcf-d938-415a-b32a-48936eee09ba" path="/var/lib/kubelet/pods/27e64bcf-d938-415a-b32a-48936eee09ba/volumes"
Sep 10 15:39:23.956820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:23.956809    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5e03482-6c9d-4ff4-9357-3aca0f28b803" path="/var/lib/kubelet/pods/b5e03482-6c9d-4ff4-9357-3aca0f28b803/volumes"
Sep 10 15:39:24.728439 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:24.728402    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:39:29.688159 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:29.688122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:29.751465 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:29.751426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:29.945514 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:29.945402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:30.287566 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.287485    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"]
Sep 10 15:39:30.288026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.288009    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27e64bcf-d938-415a-b32a-48936eee09ba" containerName="revoke-keys"
Sep 10 15:39:30.288026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.288027    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="27e64bcf-d938-415a-b32a-48936eee09ba" containerName="revoke-keys"
Sep 10 15:39:30.288177 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.288045    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5e03482-6c9d-4ff4-9357-3aca0f28b803" containerName="payload-pre-processing"
Sep 10 15:39:30.288177 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.288051    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5e03482-6c9d-4ff4-9357-3aca0f28b803" containerName="payload-pre-processing"
Sep 10 15:39:30.288177 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.288124    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="27e64bcf-d938-415a-b32a-48936eee09ba" containerName="revoke-keys"
Sep 10 15:39:30.288177 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.288134    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5e03482-6c9d-4ff4-9357-3aca0f28b803" containerName="payload-pre-processing"
Sep 10 15:39:30.290771 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.290748    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"
Sep 10 15:39:30.293195 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.293171    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:39:30.293195 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.293184    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:39:30.301071 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.301046    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"]
Sep 10 15:39:30.362226 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.362186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f02c1-56c5-4da6-a914-f8cddde12a51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb\" (UID: \"ad7f02c1-56c5-4da6-a914-f8cddde12a51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"
Sep 10 15:39:30.389393 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.389339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:30.463892 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.463850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f02c1-56c5-4da6-a914-f8cddde12a51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb\" (UID: \"ad7f02c1-56c5-4da6-a914-f8cddde12a51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"
Sep 10 15:39:30.464591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.464568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f02c1-56c5-4da6-a914-f8cddde12a51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb\" (UID: \"ad7f02c1-56c5-4da6-a914-f8cddde12a51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"
Sep 10 15:39:30.604197 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.604160    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"
Sep 10 15:39:30.877101 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:30.877072    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"]
Sep 10 15:39:30.878968 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:39:30.878930    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7f02c1_56c5_4da6_a914_f8cddde12a51.slice/crio-896caf50be9035f4d0de9e0cd38019c05e29e50666ff611ecd63bc0561d939fc WatchSource:0}: Error finding container 896caf50be9035f4d0de9e0cd38019c05e29e50666ff611ecd63bc0561d939fc: Status 404 returned error can't find the container with id 896caf50be9035f4d0de9e0cd38019c05e29e50666ff611ecd63bc0561d939fc
Sep 10 15:39:31.297206 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:31.297168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb" event={"ID":"ad7f02c1-56c5-4da6-a914-f8cddde12a51","Type":"ContainerStarted","Data":"15aee816b9bf8e09b0aaa4c4d5f7e4ee3c74297f27882d13cdd3d980b334aeff"}
Sep 10 15:39:31.297393 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:31.297216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb" event={"ID":"ad7f02c1-56c5-4da6-a914-f8cddde12a51","Type":"ContainerStarted","Data":"896caf50be9035f4d0de9e0cd38019c05e29e50666ff611ecd63bc0561d939fc"}
Sep 10 15:39:31.313103 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:31.313053    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb" podStartSLOduration=1.313037906 podStartE2EDuration="1.313037906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:39:31.312581635 +0000 UTC m=+1411.848537199" watchObservedRunningTime="2026-09-10 15:39:31.313037906 +0000 UTC m=+1411.848993512"
Sep 10 15:39:32.305090 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:32.305052    2562 generic.go:356] "Generic (PLEG): container finished" podID="ad7f02c1-56c5-4da6-a914-f8cddde12a51" containerID="15aee816b9bf8e09b0aaa4c4d5f7e4ee3c74297f27882d13cdd3d980b334aeff" exitCode=0
Sep 10 15:39:32.305090 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:32.305091    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb" event={"ID":"ad7f02c1-56c5-4da6-a914-f8cddde12a51","Type":"ContainerDied","Data":"15aee816b9bf8e09b0aaa4c4d5f7e4ee3c74297f27882d13cdd3d980b334aeff"}
Sep 10 15:39:33.279866 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:33.279831    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"]
Sep 10 15:39:33.299122 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:33.299085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:33.443091 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:33.443062    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"
Sep 10 15:39:33.593711 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:33.593677    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f02c1-56c5-4da6-a914-f8cddde12a51-maas-api-service-ca\") pod \"ad7f02c1-56c5-4da6-a914-f8cddde12a51\" (UID: \"ad7f02c1-56c5-4da6-a914-f8cddde12a51\") "
Sep 10 15:39:33.594030 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:33.594008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad7f02c1-56c5-4da6-a914-f8cddde12a51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad7f02c1-56c5-4da6-a914-f8cddde12a51" (UID: "ad7f02c1-56c5-4da6-a914-f8cddde12a51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:39:33.695112 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:33.695077    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7f02c1-56c5-4da6-a914-f8cddde12a51-maas-api-service-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:33.997667 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:39:33.997636    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7f02c1_56c5_4da6_a914_f8cddde12a51.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 15:39:34.262531 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:34.262433    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"]
Sep 10 15:39:34.283733 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:34.283702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:34.317573 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:34.317544    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"
Sep 10 15:39:34.317712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:34.317573    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb" event={"ID":"ad7f02c1-56c5-4da6-a914-f8cddde12a51","Type":"ContainerDied","Data":"896caf50be9035f4d0de9e0cd38019c05e29e50666ff611ecd63bc0561d939fc"}
Sep 10 15:39:34.317712 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:34.317615    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="896caf50be9035f4d0de9e0cd38019c05e29e50666ff611ecd63bc0561d939fc"
Sep 10 15:39:39.428251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:39.428210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:39.489221 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:39.489193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:44.422971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:44.422936    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:39:45.813692 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:45.813658    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:45.849601 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:45.849568    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"]
Sep 10 15:39:45.849860 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:45.849836    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w" podUID="15fd3bc5-1297-418a-99e4-189b8e7aad62" containerName="payload-pre-processing" containerID="cri-o://95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f" gracePeriod=30
Sep 10 15:39:45.914477 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:45.914433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:46.013744 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.013709    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"]
Sep 10 15:39:46.015138 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.015103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:46.016836 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.016812    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bbc967-hf5vb"]
Sep 10 15:39:46.115131 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.115099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:46.120837 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.120815    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:39:46.217423 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.217390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frlw4\" (UniqueName: \"kubernetes.io/projected/15fd3bc5-1297-418a-99e4-189b8e7aad62-kube-api-access-frlw4\") pod \"15fd3bc5-1297-418a-99e4-189b8e7aad62\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") "
Sep 10 15:39:46.217656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.217530    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fd3bc5-1297-418a-99e4-189b8e7aad62-plugins-config-volume\") pod \"15fd3bc5-1297-418a-99e4-189b8e7aad62\" (UID: \"15fd3bc5-1297-418a-99e4-189b8e7aad62\") "
Sep 10 15:39:46.217888 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.217863    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15fd3bc5-1297-418a-99e4-189b8e7aad62-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15fd3bc5-1297-418a-99e4-189b8e7aad62" (UID: "15fd3bc5-1297-418a-99e4-189b8e7aad62"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:39:46.219548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.219517    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15fd3bc5-1297-418a-99e4-189b8e7aad62-kube-api-access-frlw4" (OuterVolumeSpecName: "kube-api-access-frlw4") pod "15fd3bc5-1297-418a-99e4-189b8e7aad62" (UID: "15fd3bc5-1297-418a-99e4-189b8e7aad62"). InnerVolumeSpecName "kube-api-access-frlw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:39:46.318547 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.318514    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fd3bc5-1297-418a-99e4-189b8e7aad62-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:46.318547 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.318539    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frlw4\" (UniqueName: \"kubernetes.io/projected/15fd3bc5-1297-418a-99e4-189b8e7aad62-kube-api-access-frlw4\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:46.382719 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.382637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:46.387553 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.387525    2562 generic.go:356] "Generic (PLEG): container finished" podID="15fd3bc5-1297-418a-99e4-189b8e7aad62" containerID="95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f" exitCode=0
Sep 10 15:39:46.387687 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.387572    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"
Sep 10 15:39:46.387687 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.387614    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w" event={"ID":"15fd3bc5-1297-418a-99e4-189b8e7aad62","Type":"ContainerDied","Data":"95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f"}
Sep 10 15:39:46.387798 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.387704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w" event={"ID":"15fd3bc5-1297-418a-99e4-189b8e7aad62","Type":"ContainerDied","Data":"71ff2ef82908e6a8ef6e486dea9221b7237ee8bf835baee4aeefccf72231c4d3"}
Sep 10 15:39:46.387798 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.387741    2562 scope.go:117] "RemoveContainer" containerID="95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f"
Sep 10 15:39:46.397200 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.397182    2562 scope.go:117] "RemoveContainer" containerID="95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f"
Sep 10 15:39:46.397497 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:39:46.397480    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f\": container with ID starting with 95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f not found: ID does not exist" containerID="95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f"
Sep 10 15:39:46.397558 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.397505    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f"} err="failed to get container status \"95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f\": rpc error: code = NotFound desc = could not find container \"95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f\": container with ID starting with 95a2ed7998dede7d5cb30b2af6a940ed151f774a039ec2a2edb3b8b57b27e16f not found: ID does not exist"
Sep 10 15:39:46.412322 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.412301    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"]
Sep 10 15:39:46.416227 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:46.416204    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bbc967-86cf6f686b-lwb9w"]
Sep 10 15:39:47.955571 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:47.955538    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15fd3bc5-1297-418a-99e4-189b8e7aad62" path="/var/lib/kubelet/pods/15fd3bc5-1297-418a-99e4-189b8e7aad62/volumes"
Sep 10 15:39:47.955938 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:47.955904    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad7f02c1-56c5-4da6-a914-f8cddde12a51" path="/var/lib/kubelet/pods/ad7f02c1-56c5-4da6-a914-f8cddde12a51/volumes"
Sep 10 15:39:48.300388 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.300303    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz" podUID="d5f3202b-89fd-41e7-8661-9abb4c419df5" containerName="main" containerID="cri-o://91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9" gracePeriod=45
Sep 10 15:39:48.545398 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.545372    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:39:48.639665 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.639639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-tmp-dir\") pod \"d5f3202b-89fd-41e7-8661-9abb4c419df5\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") "
Sep 10 15:39:48.639820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.639707    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f3202b-89fd-41e7-8661-9abb4c419df5-tls-certs\") pod \"d5f3202b-89fd-41e7-8661-9abb4c419df5\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") "
Sep 10 15:39:48.639820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.639729    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-home\") pod \"d5f3202b-89fd-41e7-8661-9abb4c419df5\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") "
Sep 10 15:39:48.639820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.639805    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-model-cache\") pod \"d5f3202b-89fd-41e7-8661-9abb4c419df5\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") "
Sep 10 15:39:48.639971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.639839    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4xgc\" (UniqueName: \"kubernetes.io/projected/d5f3202b-89fd-41e7-8661-9abb4c419df5-kube-api-access-k4xgc\") pod \"d5f3202b-89fd-41e7-8661-9abb4c419df5\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") "
Sep 10 15:39:48.639971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.639881    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-dshm\") pod \"d5f3202b-89fd-41e7-8661-9abb4c419df5\" (UID: \"d5f3202b-89fd-41e7-8661-9abb4c419df5\") "
Sep 10 15:39:48.639971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.639895    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5f3202b-89fd-41e7-8661-9abb4c419df5" (UID: "d5f3202b-89fd-41e7-8661-9abb4c419df5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:48.640134 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.640086    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5f3202b-89fd-41e7-8661-9abb4c419df5" (UID: "d5f3202b-89fd-41e7-8661-9abb4c419df5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:48.640134 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.640098    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-home" (OuterVolumeSpecName: "home") pod "d5f3202b-89fd-41e7-8661-9abb4c419df5" (UID: "d5f3202b-89fd-41e7-8661-9abb4c419df5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:48.640316 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.640292    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:48.640316 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.640315    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:48.640529 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.640327    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:48.641980 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.641952    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f3202b-89fd-41e7-8661-9abb4c419df5-kube-api-access-k4xgc" (OuterVolumeSpecName: "kube-api-access-k4xgc") pod "d5f3202b-89fd-41e7-8661-9abb4c419df5" (UID: "d5f3202b-89fd-41e7-8661-9abb4c419df5"). InnerVolumeSpecName "kube-api-access-k4xgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:39:48.641980 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.641958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-dshm" (OuterVolumeSpecName: "dshm") pod "d5f3202b-89fd-41e7-8661-9abb4c419df5" (UID: "d5f3202b-89fd-41e7-8661-9abb4c419df5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:48.642094 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.642018    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5f3202b-89fd-41e7-8661-9abb4c419df5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5f3202b-89fd-41e7-8661-9abb4c419df5" (UID: "d5f3202b-89fd-41e7-8661-9abb4c419df5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:39:48.741026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.740998    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4xgc\" (UniqueName: \"kubernetes.io/projected/d5f3202b-89fd-41e7-8661-9abb4c419df5-kube-api-access-k4xgc\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:48.741026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.741022    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f3202b-89fd-41e7-8661-9abb4c419df5-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:48.741026 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:48.741031    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f3202b-89fd-41e7-8661-9abb4c419df5-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:49.282357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.282317    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m" podUID="dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" containerName="main" containerID="cri-o://3552ebe131d39f43a2ac3855dcb2ee696b72ffcbff7dd01a4721dc7bd84aaa5a" gracePeriod=45
Sep 10 15:39:49.407238 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.407210    2562 generic.go:356] "Generic (PLEG): container finished" podID="d5f3202b-89fd-41e7-8661-9abb4c419df5" containerID="91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9" exitCode=0
Sep 10 15:39:49.407360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.407251    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"
Sep 10 15:39:49.407360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.407294    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz" event={"ID":"d5f3202b-89fd-41e7-8661-9abb4c419df5","Type":"ContainerDied","Data":"91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9"}
Sep 10 15:39:49.407360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.407334    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz" event={"ID":"d5f3202b-89fd-41e7-8661-9abb4c419df5","Type":"ContainerDied","Data":"b94e7152d18bc32dd1de5d5024b84aab6c14428c1ed7e543b9a047b64806f64f"}
Sep 10 15:39:49.407360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.407352    2562 scope.go:117] "RemoveContainer" containerID="91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9"
Sep 10 15:39:49.409241 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.409208    2562 generic.go:356] "Generic (PLEG): container finished" podID="dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" containerID="3552ebe131d39f43a2ac3855dcb2ee696b72ffcbff7dd01a4721dc7bd84aaa5a" exitCode=0
Sep 10 15:39:49.409352 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.409283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m" event={"ID":"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db","Type":"ContainerDied","Data":"3552ebe131d39f43a2ac3855dcb2ee696b72ffcbff7dd01a4721dc7bd84aaa5a"}
Sep 10 15:39:49.416504 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.416486    2562 scope.go:117] "RemoveContainer" containerID="91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9"
Sep 10 15:39:49.416776 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:39:49.416753    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9\": container with ID starting with 91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9 not found: ID does not exist" containerID="91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9"
Sep 10 15:39:49.416866 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.416786    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9"} err="failed to get container status \"91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9\": rpc error: code = NotFound desc = could not find container \"91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9\": container with ID starting with 91863da30cc0d2c20dc9935bfdfd9dfac566331abe0cf1a71e2114a07375fed9 not found: ID does not exist"
Sep 10 15:39:49.431147 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.431119    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"]
Sep 10 15:39:49.435714 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.435689    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-62d2f7/sub-test-model-62d2f7-kserve-78d978ff65-t27hz"]
Sep 10 15:39:49.544653 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.544595    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:39:49.650468 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-dshm\") pod \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") "
Sep 10 15:39:49.650623 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650509    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tmp-dir\") pod \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") "
Sep 10 15:39:49.650623 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650551    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-home\") pod \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") "
Sep 10 15:39:49.650694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650628    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tls-certs\") pod \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") "
Sep 10 15:39:49.650728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650698    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pgdv\" (UniqueName: \"kubernetes.io/projected/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-kube-api-access-2pgdv\") pod \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") "
Sep 10 15:39:49.650782 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650737    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-model-cache\") pod \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\" (UID: \"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db\") "
Sep 10 15:39:49.650834 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650790    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" (UID: "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:49.650834 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.650800    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-home" (OuterVolumeSpecName: "home") pod "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" (UID: "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:49.651060 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.651036    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-model-cache" (OuterVolumeSpecName: "model-cache") pod "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" (UID: "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:49.651060 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.651049    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:49.651207 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.651077    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:49.652708 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.652682    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" (UID: "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:39:49.652820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.652717    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-dshm" (OuterVolumeSpecName: "dshm") pod "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" (UID: "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:39:49.652876 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.652826    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-kube-api-access-2pgdv" (OuterVolumeSpecName: "kube-api-access-2pgdv") pod "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" (UID: "dc0f7596-2d6c-4ff9-8d65-54c823b6c7db"). InnerVolumeSpecName "kube-api-access-2pgdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:39:49.751870 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.751833    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:49.751870 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.751863    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:49.751870 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.751872    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:49.752078 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.751880    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pgdv\" (UniqueName: \"kubernetes.io/projected/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db-kube-api-access-2pgdv\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:39:49.955268 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:49.955225    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5f3202b-89fd-41e7-8661-9abb4c419df5" path="/var/lib/kubelet/pods/d5f3202b-89fd-41e7-8661-9abb4c419df5/volumes"
Sep 10 15:39:50.416852 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:50.416816    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m" event={"ID":"dc0f7596-2d6c-4ff9-8d65-54c823b6c7db","Type":"ContainerDied","Data":"8338fab3267352688de808f26b675a5d300b5e0465b2075f9d99fa98f4d8ffd9"}
Sep 10 15:39:50.416852 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:50.416852    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"
Sep 10 15:39:50.417326 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:50.416871    2562 scope.go:117] "RemoveContainer" containerID="3552ebe131d39f43a2ac3855dcb2ee696b72ffcbff7dd01a4721dc7bd84aaa5a"
Sep 10 15:39:50.434405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:50.434381    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"]
Sep 10 15:39:50.436267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:50.436247    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/sub-test-model-fc3e54-kserve-6b8d8fd8b7-brr6m"]
Sep 10 15:39:51.955897 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:51.955864    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" path="/var/lib/kubelet/pods/dc0f7596-2d6c-4ff9-8d65-54c823b6c7db/volumes"
Sep 10 15:39:52.619745 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:52.619709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:52.682233 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:52.682204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:52.760186 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:52.760156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:52.887217 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:52.887132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:53.470022 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:53.469985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:56.833008 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:39:56.832970    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:40:03.771210 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:03.771168    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"]
Sep 10 15:40:03.771794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:03.771412    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp" podUID="d636ff7a-a331-4cf2-b930-8013fce2d858" containerName="payload-processing" containerID="cri-o://19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53" gracePeriod=30
Sep 10 15:40:03.816361 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:03.816328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:03.960782 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:03.960746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:04.024874 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.024809    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:40:04.025278 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.025256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:04.099017 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.098987    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d636ff7a-a331-4cf2-b930-8013fce2d858-plugins-config-volume\") pod \"d636ff7a-a331-4cf2-b930-8013fce2d858\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") "
Sep 10 15:40:04.099181 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.099055    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4z4l\" (UniqueName: \"kubernetes.io/projected/d636ff7a-a331-4cf2-b930-8013fce2d858-kube-api-access-r4z4l\") pod \"d636ff7a-a331-4cf2-b930-8013fce2d858\" (UID: \"d636ff7a-a331-4cf2-b930-8013fce2d858\") "
Sep 10 15:40:04.099421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.099396    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d636ff7a-a331-4cf2-b930-8013fce2d858-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d636ff7a-a331-4cf2-b930-8013fce2d858" (UID: "d636ff7a-a331-4cf2-b930-8013fce2d858"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:40:04.101087 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.101058    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d636ff7a-a331-4cf2-b930-8013fce2d858-kube-api-access-r4z4l" (OuterVolumeSpecName: "kube-api-access-r4z4l") pod "d636ff7a-a331-4cf2-b930-8013fce2d858" (UID: "d636ff7a-a331-4cf2-b930-8013fce2d858"). InnerVolumeSpecName "kube-api-access-r4z4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:40:04.200029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.199994    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d636ff7a-a331-4cf2-b930-8013fce2d858-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:40:04.200029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.200021    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4z4l\" (UniqueName: \"kubernetes.io/projected/d636ff7a-a331-4cf2-b930-8013fce2d858-kube-api-access-r4z4l\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:40:04.273443 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.273407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:04.493294 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.493254    2562 generic.go:356] "Generic (PLEG): container finished" podID="d636ff7a-a331-4cf2-b930-8013fce2d858" containerID="19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53" exitCode=0
Sep 10 15:40:04.493496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.493310    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp" event={"ID":"d636ff7a-a331-4cf2-b930-8013fce2d858","Type":"ContainerDied","Data":"19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53"}
Sep 10 15:40:04.493496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.493318    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"
Sep 10 15:40:04.493496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.493391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp" event={"ID":"d636ff7a-a331-4cf2-b930-8013fce2d858","Type":"ContainerDied","Data":"c7e5a6482acbec336906ba1dfc537ebe3f6a8558d6eb7d3948e121df045ba8e4"}
Sep 10 15:40:04.493496 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.493421    2562 scope.go:117] "RemoveContainer" containerID="19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53"
Sep 10 15:40:04.503004 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.502978    2562 scope.go:117] "RemoveContainer" containerID="19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53"
Sep 10 15:40:04.503285 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:40:04.503265    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53\": container with ID starting with 19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53 not found: ID does not exist" containerID="19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53"
Sep 10 15:40:04.503340 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.503293    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53"} err="failed to get container status \"19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53\": rpc error: code = NotFound desc = could not find container \"19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53\": container with ID starting with 19b0afd9bb0048ad030e19920f3a1683843e01187de27de5308cbb8fa7532d53 not found: ID does not exist"
Sep 10 15:40:04.514467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.514418    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"]
Sep 10 15:40:04.517966 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:04.517938    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5fe45b-85f7569467-82zsp"]
Sep 10 15:40:05.955799 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:05.955764    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d636ff7a-a331-4cf2-b930-8013fce2d858" path="/var/lib/kubelet/pods/d636ff7a-a331-4cf2-b930-8013fce2d858/volumes"
Sep 10 15:40:07.254054 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:07.254019    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:10.420238 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:10.420205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:10.477927 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:10.477896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:10.677980 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:10.677894    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:11.111020 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:11.110984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:13.725495 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:13.725439    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:40:14.677987 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:14.677955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:19.236022 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.235989    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"]
Sep 10 15:40:19.236516 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236498    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad7f02c1-56c5-4da6-a914-f8cddde12a51" containerName="revoke-keys"
Sep 10 15:40:19.236516 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236518    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad7f02c1-56c5-4da6-a914-f8cddde12a51" containerName="revoke-keys"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236539    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" containerName="main"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236548    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" containerName="main"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236560    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d636ff7a-a331-4cf2-b930-8013fce2d858" containerName="payload-processing"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236570    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d636ff7a-a331-4cf2-b930-8013fce2d858" containerName="payload-processing"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236581    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f3202b-89fd-41e7-8661-9abb4c419df5" containerName="main"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236587    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f3202b-89fd-41e7-8661-9abb4c419df5" containerName="main"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236607    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15fd3bc5-1297-418a-99e4-189b8e7aad62" containerName="payload-pre-processing"
Sep 10 15:40:19.236642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236613    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="15fd3bc5-1297-418a-99e4-189b8e7aad62" containerName="payload-pre-processing"
Sep 10 15:40:19.236921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236715    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d636ff7a-a331-4cf2-b930-8013fce2d858" containerName="payload-processing"
Sep 10 15:40:19.236921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236729    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="15fd3bc5-1297-418a-99e4-189b8e7aad62" containerName="payload-pre-processing"
Sep 10 15:40:19.236921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236740    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc0f7596-2d6c-4ff9-8d65-54c823b6c7db" containerName="main"
Sep 10 15:40:19.236921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236754    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad7f02c1-56c5-4da6-a914-f8cddde12a51" containerName="revoke-keys"
Sep 10 15:40:19.236921 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.236763    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f3202b-89fd-41e7-8661-9abb4c419df5" containerName="main"
Sep 10 15:40:19.241170 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.241151    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.244300 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244237    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/571d82fe-ed4c-40dc-8443-0a520edfad1b-tls-certs\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.244300 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcnvr\" (UniqueName: \"kubernetes.io/projected/571d82fe-ed4c-40dc-8443-0a520edfad1b-kube-api-access-xcnvr\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.244505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-tmp-dir\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.244505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-dshm\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.244505 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-home\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.244725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244523    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"kube-root-ca.crt\""
Sep 10 15:40:19.244725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-model-cache\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.244725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244556    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"openshift-service-ca.crt\""
Sep 10 15:40:19.244725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244548    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"default-dockercfg-wvljq\""
Sep 10 15:40:19.244725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.244696    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc3e54\"/\"rate-test-model-fc3e54-kserve-self-signed-certs\""
Sep 10 15:40:19.250446 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.250426    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"]
Sep 10 15:40:19.266190 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.266170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:19.345107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/571d82fe-ed4c-40dc-8443-0a520edfad1b-tls-certs\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345107 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcnvr\" (UniqueName: \"kubernetes.io/projected/571d82fe-ed4c-40dc-8443-0a520edfad1b-kube-api-access-xcnvr\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-tmp-dir\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-dshm\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-home\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-model-cache\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345700 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-tmp-dir\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345764 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-home\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.345764 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.345733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-model-cache\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.347370 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.347347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-dshm\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.347590 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.347575    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/571d82fe-ed4c-40dc-8443-0a520edfad1b-tls-certs\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.354175 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.354159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcnvr\" (UniqueName: \"kubernetes.io/projected/571d82fe-ed4c-40dc-8443-0a520edfad1b-kube-api-access-xcnvr\") pod \"rate-test-model-fc3e54-kserve-56f869d4c-phdrs\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") " pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.552299 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.552204    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:19.826623 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:19.826599    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"]
Sep 10 15:40:19.828844 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:40:19.828817    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod571d82fe_ed4c_40dc_8443_0a520edfad1b.slice/crio-6b9361e18e52a07ab2388f56afecd94b1f25fa4d8595a4893f699e1c6c282636 WatchSource:0}: Error finding container 6b9361e18e52a07ab2388f56afecd94b1f25fa4d8595a4893f699e1c6c282636: Status 404 returned error can't find the container with id 6b9361e18e52a07ab2388f56afecd94b1f25fa4d8595a4893f699e1c6c282636
Sep 10 15:40:20.579824 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:20.579783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs" event={"ID":"571d82fe-ed4c-40dc-8443-0a520edfad1b","Type":"ContainerStarted","Data":"ff54f57dc10bc7c267c2cf995adf91a1333369ca5460f216f220a27b918f6721"}
Sep 10 15:40:20.580263 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:20.579831    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs" event={"ID":"571d82fe-ed4c-40dc-8443-0a520edfad1b","Type":"ContainerStarted","Data":"6b9361e18e52a07ab2388f56afecd94b1f25fa4d8595a4893f699e1c6c282636"}
Sep 10 15:40:20.599074 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:20.599024    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs" podStartSLOduration=1.599011612 podStartE2EDuration="1.599011612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:40:20.596877008 +0000 UTC m=+1461.132832597" watchObservedRunningTime="2026-09-10 15:40:20.599011612 +0000 UTC m=+1461.134967174"
Sep 10 15:40:21.245317 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.245270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:21.810620 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.810570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:21.825980 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.825951    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"]
Sep 10 15:40:21.826202 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.826177    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d" podUID="03154feb-1cfa-4764-9a67-a2ecc972cc00" containerName="payload-processing" containerID="cri-o://0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486" gracePeriod=30
Sep 10 15:40:21.831193 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.831165    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"]
Sep 10 15:40:21.831414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.831390    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74" podUID="42a76ea2-4e85-4017-bf3a-de8546983c90" containerName="payload-pre-processing" containerID="cri-o://885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543" gracePeriod=30
Sep 10 15:40:21.899382 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.899357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:21.972110 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:21.972067    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:22.029119 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.029095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:22.104157 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.104137    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:40:22.106686 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.106670    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:40:22.172702 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.172668    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kxfg\" (UniqueName: \"kubernetes.io/projected/42a76ea2-4e85-4017-bf3a-de8546983c90-kube-api-access-7kxfg\") pod \"42a76ea2-4e85-4017-bf3a-de8546983c90\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") "
Sep 10 15:40:22.172894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.172742    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl5xm\" (UniqueName: \"kubernetes.io/projected/03154feb-1cfa-4764-9a67-a2ecc972cc00-kube-api-access-vl5xm\") pod \"03154feb-1cfa-4764-9a67-a2ecc972cc00\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") "
Sep 10 15:40:22.172894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.172774    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03154feb-1cfa-4764-9a67-a2ecc972cc00-plugins-config-volume\") pod \"03154feb-1cfa-4764-9a67-a2ecc972cc00\" (UID: \"03154feb-1cfa-4764-9a67-a2ecc972cc00\") "
Sep 10 15:40:22.172894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.172830    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a76ea2-4e85-4017-bf3a-de8546983c90-plugins-config-volume\") pod \"42a76ea2-4e85-4017-bf3a-de8546983c90\" (UID: \"42a76ea2-4e85-4017-bf3a-de8546983c90\") "
Sep 10 15:40:22.173331 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.173298    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a76ea2-4e85-4017-bf3a-de8546983c90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42a76ea2-4e85-4017-bf3a-de8546983c90" (UID: "42a76ea2-4e85-4017-bf3a-de8546983c90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:40:22.173425 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.173337    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03154feb-1cfa-4764-9a67-a2ecc972cc00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03154feb-1cfa-4764-9a67-a2ecc972cc00" (UID: "03154feb-1cfa-4764-9a67-a2ecc972cc00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:40:22.174891 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.174869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03154feb-1cfa-4764-9a67-a2ecc972cc00-kube-api-access-vl5xm" (OuterVolumeSpecName: "kube-api-access-vl5xm") pod "03154feb-1cfa-4764-9a67-a2ecc972cc00" (UID: "03154feb-1cfa-4764-9a67-a2ecc972cc00"). InnerVolumeSpecName "kube-api-access-vl5xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:40:22.174978 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.174941    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a76ea2-4e85-4017-bf3a-de8546983c90-kube-api-access-7kxfg" (OuterVolumeSpecName: "kube-api-access-7kxfg") pod "42a76ea2-4e85-4017-bf3a-de8546983c90" (UID: "42a76ea2-4e85-4017-bf3a-de8546983c90"). InnerVolumeSpecName "kube-api-access-7kxfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:40:22.273846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.273816    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kxfg\" (UniqueName: \"kubernetes.io/projected/42a76ea2-4e85-4017-bf3a-de8546983c90-kube-api-access-7kxfg\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:40:22.273846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.273842    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl5xm\" (UniqueName: \"kubernetes.io/projected/03154feb-1cfa-4764-9a67-a2ecc972cc00-kube-api-access-vl5xm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:40:22.273991 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.273851    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03154feb-1cfa-4764-9a67-a2ecc972cc00-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:40:22.273991 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.273861    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a76ea2-4e85-4017-bf3a-de8546983c90-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:40:22.319610 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.319579    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:22.591389 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.591346    2562 generic.go:356] "Generic (PLEG): container finished" podID="42a76ea2-4e85-4017-bf3a-de8546983c90" containerID="885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543" exitCode=0
Sep 10 15:40:22.591569 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.591395    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"
Sep 10 15:40:22.591569 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.591430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74" event={"ID":"42a76ea2-4e85-4017-bf3a-de8546983c90","Type":"ContainerDied","Data":"885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543"}
Sep 10 15:40:22.591569 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.591509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74" event={"ID":"42a76ea2-4e85-4017-bf3a-de8546983c90","Type":"ContainerDied","Data":"81d2b2f8887559d1147d5a489ec9b3bb131a44a44e3a42033ce7613ae73388b8"}
Sep 10 15:40:22.591569 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.591541    2562 scope.go:117] "RemoveContainer" containerID="885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543"
Sep 10 15:40:22.592769 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.592740    2562 generic.go:356] "Generic (PLEG): container finished" podID="03154feb-1cfa-4764-9a67-a2ecc972cc00" containerID="0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486" exitCode=0
Sep 10 15:40:22.592897 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.592774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d" event={"ID":"03154feb-1cfa-4764-9a67-a2ecc972cc00","Type":"ContainerDied","Data":"0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486"}
Sep 10 15:40:22.592897 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.592794    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"
Sep 10 15:40:22.592897 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.592834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d" event={"ID":"03154feb-1cfa-4764-9a67-a2ecc972cc00","Type":"ContainerDied","Data":"c62a1ca70cbceb0cacee9c78da01ffb9261d04b8ad505e2e3a96e7f0411a2070"}
Sep 10 15:40:22.601392 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.601378    2562 scope.go:117] "RemoveContainer" containerID="885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543"
Sep 10 15:40:22.601686 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:40:22.601665    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543\": container with ID starting with 885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543 not found: ID does not exist" containerID="885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543"
Sep 10 15:40:22.601775 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.601696    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543"} err="failed to get container status \"885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543\": rpc error: code = NotFound desc = could not find container \"885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543\": container with ID starting with 885a35691764769a2b58fa3b2b561677a6e2911391fd9af6ce061750c5d2e543 not found: ID does not exist"
Sep 10 15:40:22.601775 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.601717    2562 scope.go:117] "RemoveContainer" containerID="0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486"
Sep 10 15:40:22.609946 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.609931    2562 scope.go:117] "RemoveContainer" containerID="0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486"
Sep 10 15:40:22.610193 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:40:22.610173    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486\": container with ID starting with 0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486 not found: ID does not exist" containerID="0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486"
Sep 10 15:40:22.610248 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.610199    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486"} err="failed to get container status \"0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486\": rpc error: code = NotFound desc = could not find container \"0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486\": container with ID starting with 0f9aa20c693a969cddc626afd2c5fc32c894f6f0ac9cf0e58035beb81e8f8486 not found: ID does not exist"
Sep 10 15:40:22.616079 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.616059    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"]
Sep 10 15:40:22.620011 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.619991    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d7a2b3-867fc5cb5-fl47d"]
Sep 10 15:40:22.630173 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.630153    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"]
Sep 10 15:40:22.633694 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:22.633673    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d7a2b3-5bb85f6ccd-j6f74"]
Sep 10 15:40:23.956725 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:23.956684    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03154feb-1cfa-4764-9a67-a2ecc972cc00" path="/var/lib/kubelet/pods/03154feb-1cfa-4764-9a67-a2ecc972cc00/volumes"
Sep 10 15:40:23.957291 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:23.957269    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42a76ea2-4e85-4017-bf3a-de8546983c90" path="/var/lib/kubelet/pods/42a76ea2-4e85-4017-bf3a-de8546983c90/volumes"
Sep 10 15:40:28.469013 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:28.468983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:28.527592 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:28.527557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:28.725178 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:28.725090    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:29.553160 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:29.553123    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:29.553570 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:29.553172    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:29.565640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:29.565616    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:29.646146 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:29.646113    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:40:30.427643 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:30.427613    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:40:44.043323 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:44.043288    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:40:54.723314 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:40:54.723276    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:41:00.001824 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:00.001790    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:41:00.006142 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:00.006122    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:41:00.653914 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:00.653878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:00.718832 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:00.718802    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"]
Sep 10 15:41:00.738979 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:00.738948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:01.147468 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:01.147429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:01.446578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:01.446492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:01.699072 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:01.698989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:04.270093 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:04.270064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:04.370405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:04.370372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:05.857640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:05.857608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:05.959682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:05.959642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:06.030882 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.030844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:06.035823 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.035797    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"]
Sep 10 15:41:06.036205 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.036191    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03154feb-1cfa-4764-9a67-a2ecc972cc00" containerName="payload-processing"
Sep 10 15:41:06.036205 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.036205    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="03154feb-1cfa-4764-9a67-a2ecc972cc00" containerName="payload-processing"
Sep 10 15:41:06.036293 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.036226    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a76ea2-4e85-4017-bf3a-de8546983c90" containerName="payload-pre-processing"
Sep 10 15:41:06.036293 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.036233    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a76ea2-4e85-4017-bf3a-de8546983c90" containerName="payload-pre-processing"
Sep 10 15:41:06.036361 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.036305    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a76ea2-4e85-4017-bf3a-de8546983c90" containerName="payload-pre-processing"
Sep 10 15:41:06.036361 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.036316    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="03154feb-1cfa-4764-9a67-a2ecc972cc00" containerName="payload-processing"
Sep 10 15:41:06.040351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.040335    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.042971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.042954    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-ba5c7e\""
Sep 10 15:41:06.043065 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.042955    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-ba5c7e-dockercfg-fwm9l\""
Sep 10 15:41:06.048749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.048726    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"]
Sep 10 15:41:06.079495 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.079445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.079613 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.079511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzx4\" (UniqueName: \"kubernetes.io/projected/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-kube-api-access-4tzx4\") pod \"payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.180043 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.179967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.180043 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.180011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzx4\" (UniqueName: \"kubernetes.io/projected/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-kube-api-access-4tzx4\") pod \"payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.180654 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.180633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.187551 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.187529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzx4\" (UniqueName: \"kubernetes.io/projected/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-kube-api-access-4tzx4\") pod \"payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.352344 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.352308    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.627114 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.627088    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"]
Sep 10 15:41:06.629153 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:41:06.629117    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09804b5_64d8_45b8_9ede_6dc1a0f7a791.slice/crio-36a7322deb5e349af061543ec3ff65626f35ff39722052e50225ae33005e7004 WatchSource:0}: Error finding container 36a7322deb5e349af061543ec3ff65626f35ff39722052e50225ae33005e7004: Status 404 returned error can't find the container with id 36a7322deb5e349af061543ec3ff65626f35ff39722052e50225ae33005e7004
Sep 10 15:41:06.836010 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.835976    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf" event={"ID":"e09804b5-64d8-45b8-9ede-6dc1a0f7a791","Type":"ContainerStarted","Data":"618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa"}
Sep 10 15:41:06.836217 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.836034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf" event={"ID":"e09804b5-64d8-45b8-9ede-6dc1a0f7a791","Type":"ContainerStarted","Data":"36a7322deb5e349af061543ec3ff65626f35ff39722052e50225ae33005e7004"}
Sep 10 15:41:06.836217 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.836061    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:06.854246 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:06.854192    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf" podStartSLOduration=0.854176016 podStartE2EDuration="854.176016ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:06.852187285 +0000 UTC m=+1507.388142867" watchObservedRunningTime="2026-09-10 15:41:06.854176016 +0000 UTC m=+1507.390131620"
Sep 10 15:41:08.637412 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:08.637365    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:41:15.738948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:15.738878    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs" podUID="571d82fe-ed4c-40dc-8443-0a520edfad1b" containerName="main" containerID="cri-o://ff54f57dc10bc7c267c2cf995adf91a1333369ca5460f216f220a27b918f6721" gracePeriod=45
Sep 10 15:41:15.888763 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:15.888723    2562 generic.go:356] "Generic (PLEG): container finished" podID="571d82fe-ed4c-40dc-8443-0a520edfad1b" containerID="ff54f57dc10bc7c267c2cf995adf91a1333369ca5460f216f220a27b918f6721" exitCode=0
Sep 10 15:41:15.888968 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:15.888801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs" event={"ID":"571d82fe-ed4c-40dc-8443-0a520edfad1b","Type":"ContainerDied","Data":"ff54f57dc10bc7c267c2cf995adf91a1333369ca5460f216f220a27b918f6721"}
Sep 10 15:41:16.000360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.000288    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:41:16.076841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.076807    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-model-cache\") pod \"571d82fe-ed4c-40dc-8443-0a520edfad1b\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") "
Sep 10 15:41:16.077021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.076851    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-tmp-dir\") pod \"571d82fe-ed4c-40dc-8443-0a520edfad1b\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") "
Sep 10 15:41:16.077021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.076953    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcnvr\" (UniqueName: \"kubernetes.io/projected/571d82fe-ed4c-40dc-8443-0a520edfad1b-kube-api-access-xcnvr\") pod \"571d82fe-ed4c-40dc-8443-0a520edfad1b\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") "
Sep 10 15:41:16.077021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077009    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/571d82fe-ed4c-40dc-8443-0a520edfad1b-tls-certs\") pod \"571d82fe-ed4c-40dc-8443-0a520edfad1b\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") "
Sep 10 15:41:16.077163 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-dshm\") pod \"571d82fe-ed4c-40dc-8443-0a520edfad1b\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") "
Sep 10 15:41:16.077163 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077078    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-home\") pod \"571d82fe-ed4c-40dc-8443-0a520edfad1b\" (UID: \"571d82fe-ed4c-40dc-8443-0a520edfad1b\") "
Sep 10 15:41:16.077163 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077106    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "571d82fe-ed4c-40dc-8443-0a520edfad1b" (UID: "571d82fe-ed4c-40dc-8443-0a520edfad1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:41:16.077324 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077182    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "571d82fe-ed4c-40dc-8443-0a520edfad1b" (UID: "571d82fe-ed4c-40dc-8443-0a520edfad1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:41:16.077386 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077354    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-home" (OuterVolumeSpecName: "home") pod "571d82fe-ed4c-40dc-8443-0a520edfad1b" (UID: "571d82fe-ed4c-40dc-8443-0a520edfad1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:41:16.077473 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077429    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-model-cache\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:41:16.077522 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.077493    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-tmp-dir\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:41:16.079165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.079132    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-dshm" (OuterVolumeSpecName: "dshm") pod "571d82fe-ed4c-40dc-8443-0a520edfad1b" (UID: "571d82fe-ed4c-40dc-8443-0a520edfad1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:41:16.079281 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.079168    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/571d82fe-ed4c-40dc-8443-0a520edfad1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "571d82fe-ed4c-40dc-8443-0a520edfad1b" (UID: "571d82fe-ed4c-40dc-8443-0a520edfad1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:41:16.079281 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.079168    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/571d82fe-ed4c-40dc-8443-0a520edfad1b-kube-api-access-xcnvr" (OuterVolumeSpecName: "kube-api-access-xcnvr") pod "571d82fe-ed4c-40dc-8443-0a520edfad1b" (UID: "571d82fe-ed4c-40dc-8443-0a520edfad1b"). InnerVolumeSpecName "kube-api-access-xcnvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:41:16.178770 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.178730    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcnvr\" (UniqueName: \"kubernetes.io/projected/571d82fe-ed4c-40dc-8443-0a520edfad1b-kube-api-access-xcnvr\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:41:16.178770 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.178768    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/571d82fe-ed4c-40dc-8443-0a520edfad1b-tls-certs\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:41:16.178770 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.178781    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-dshm\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:41:16.179025 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.178791    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/571d82fe-ed4c-40dc-8443-0a520edfad1b-home\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:41:16.895037 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.895009    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"
Sep 10 15:41:16.895568 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.895004    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs" event={"ID":"571d82fe-ed4c-40dc-8443-0a520edfad1b","Type":"ContainerDied","Data":"6b9361e18e52a07ab2388f56afecd94b1f25fa4d8595a4893f699e1c6c282636"}
Sep 10 15:41:16.895568 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.895134    2562 scope.go:117] "RemoveContainer" containerID="ff54f57dc10bc7c267c2cf995adf91a1333369ca5460f216f220a27b918f6721"
Sep 10 15:41:16.917633 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.917599    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"]
Sep 10 15:41:16.921509 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:16.921485    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc3e54/rate-test-model-fc3e54-kserve-56f869d4c-phdrs"]
Sep 10 15:41:17.844597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:17.844565    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:41:17.955950 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:17.955918    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="571d82fe-ed4c-40dc-8443-0a520edfad1b" path="/var/lib/kubelet/pods/571d82fe-ed4c-40dc-8443-0a520edfad1b/volumes"
Sep 10 15:41:29.181686 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:29.181608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:29.834153 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:29.834115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:31.356143 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:31.356105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:31.457420 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:31.457378    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:33.413740 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:33.413700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:33.505217 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:33.505182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:34.556109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:34.556067    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:34.656155 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:34.656118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:35.057245 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.057204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:35.186822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.186783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:35.323463 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.323429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:35.356351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.356259    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"]
Sep 10 15:41:35.356946 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.356920    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="571d82fe-ed4c-40dc-8443-0a520edfad1b" containerName="main"
Sep 10 15:41:35.356946 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.356948    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="571d82fe-ed4c-40dc-8443-0a520edfad1b" containerName="main"
Sep 10 15:41:35.357082 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.357056    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="571d82fe-ed4c-40dc-8443-0a520edfad1b" containerName="main"
Sep 10 15:41:35.360284 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.360267    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.362916 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.362891    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-09b833-dockercfg-v44d8\""
Sep 10 15:41:35.363052 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.362891    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-09b833\""
Sep 10 15:41:35.371864 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.371837    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"]
Sep 10 15:41:35.481981 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.481942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.482178 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.482041    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhc2n\" (UniqueName: \"kubernetes.io/projected/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-kube-api-access-bhc2n\") pod \"payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.582933 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.582839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.582933 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.582912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhc2n\" (UniqueName: \"kubernetes.io/projected/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-kube-api-access-bhc2n\") pod \"payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.583677 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.583656    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.591794 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.591764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhc2n\" (UniqueName: \"kubernetes.io/projected/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-kube-api-access-bhc2n\") pod \"payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.671426 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.671381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:35.948048 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:35.948015    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"]
Sep 10 15:41:35.950885 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:41:35.950852    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded91e65c_200f_4a9f_98bc_5edd3f45ddb5.slice/crio-cfc3498b2f59639cf73ae3f29c68a9f2d295a11bb44d6622135817ee22b8eb88 WatchSource:0}: Error finding container cfc3498b2f59639cf73ae3f29c68a9f2d295a11bb44d6622135817ee22b8eb88: Status 404 returned error can't find the container with id cfc3498b2f59639cf73ae3f29c68a9f2d295a11bb44d6622135817ee22b8eb88
Sep 10 15:41:36.001467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:36.001419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb" event={"ID":"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5","Type":"ContainerStarted","Data":"cfc3498b2f59639cf73ae3f29c68a9f2d295a11bb44d6622135817ee22b8eb88"}
Sep 10 15:41:37.008280 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:37.008241    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb" event={"ID":"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5","Type":"ContainerStarted","Data":"9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d"}
Sep 10 15:41:37.008802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:37.008416    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:37.030078 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:37.030025    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb" podStartSLOduration=2.03000396 podStartE2EDuration="2.03000396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:37.028619581 +0000 UTC m=+1537.564575162" watchObservedRunningTime="2026-09-10 15:41:37.03000396 +0000 UTC m=+1537.565959521"
Sep 10 15:41:38.234943 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:38.234899    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:41:48.016704 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:48.016671    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:41:58.376639 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:41:58.376594    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:03.569114 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:03.569057    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb_ed91e65c-200f-4a9f-98bc-5edd3f45ddb5/payload-processing/0.log"
Sep 10 15:42:15.882943 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:15.882905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:15.982543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:15.982495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:20.162023 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:20.161991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:24.125996 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:24.125956    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:42:43.129207 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:43.129170    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:42:49.585437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:49.585405    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb_ed91e65c-200f-4a9f-98bc-5edd3f45ddb5/payload-processing/0.log"
Sep 10 15:42:49.865323 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:49.865198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:50.159959 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:50.159861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:50.466376 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:50.466292    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:52.891843 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:52.891758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:52.992242 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:52.992190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:54.186761 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.186719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:54.282872 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.282825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:54.383755 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.383709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:54.687425 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.687388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:54.773211 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.773141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:54.777914 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.777882    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"]
Sep 10 15:42:54.781994 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.781965    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:54.784486 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.784441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-810029\""
Sep 10 15:42:54.784614 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.784483    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-810029-dockercfg-vtpb7\""
Sep 10 15:42:54.790900 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.790872    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"]
Sep 10 15:42:54.818225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.818192    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"]
Sep 10 15:42:54.821965 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.821941    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:54.832055 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.832026    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"]
Sep 10 15:42:54.905978 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.905938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1788655a-cec8-4786-80e4-2963e93f7b6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:54.906165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.906055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsm7c\" (UniqueName: \"kubernetes.io/projected/f66fc007-88c3-4a18-8a21-b1cba09ed51c-kube-api-access-wsm7c\") pod \"payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:54.906165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.906086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrw8k\" (UniqueName: \"kubernetes.io/projected/1788655a-cec8-4786-80e4-2963e93f7b6d-kube-api-access-vrw8k\") pod \"payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:54.906255 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:54.906200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66fc007-88c3-4a18-8a21-b1cba09ed51c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:55.007679 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.007545    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsm7c\" (UniqueName: \"kubernetes.io/projected/f66fc007-88c3-4a18-8a21-b1cba09ed51c-kube-api-access-wsm7c\") pod \"payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:55.007963 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.007934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrw8k\" (UniqueName: \"kubernetes.io/projected/1788655a-cec8-4786-80e4-2963e93f7b6d-kube-api-access-vrw8k\") pod \"payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:55.008075 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.008032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66fc007-88c3-4a18-8a21-b1cba09ed51c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:55.008139 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.008079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1788655a-cec8-4786-80e4-2963e93f7b6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:55.009033 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.009009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1788655a-cec8-4786-80e4-2963e93f7b6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:55.009033 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.009026    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66fc007-88c3-4a18-8a21-b1cba09ed51c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:55.015570 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.015547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsm7c\" (UniqueName: \"kubernetes.io/projected/f66fc007-88c3-4a18-8a21-b1cba09ed51c-kube-api-access-wsm7c\") pod \"payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:55.015695 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.015677    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrw8k\" (UniqueName: \"kubernetes.io/projected/1788655a-cec8-4786-80e4-2963e93f7b6d-kube-api-access-vrw8k\") pod \"payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:55.094228 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.094170    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:55.133781 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.133741    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:55.367087 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.367052    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"]
Sep 10 15:42:55.368593 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:42:55.368554    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1788655a_cec8_4786_80e4_2963e93f7b6d.slice/crio-533d1212b4e76e9f6d23476807a29e9d1a012bd8f8e34cbc269f318cca8e813b WatchSource:0}: Error finding container 533d1212b4e76e9f6d23476807a29e9d1a012bd8f8e34cbc269f318cca8e813b: Status 404 returned error can't find the container with id 533d1212b4e76e9f6d23476807a29e9d1a012bd8f8e34cbc269f318cca8e813b
Sep 10 15:42:55.418090 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.417989    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj" event={"ID":"1788655a-cec8-4786-80e4-2963e93f7b6d","Type":"ContainerStarted","Data":"533d1212b4e76e9f6d23476807a29e9d1a012bd8f8e34cbc269f318cca8e813b"}
Sep 10 15:42:55.418820 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:55.418800    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"]
Sep 10 15:42:55.420316 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:42:55.420287    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66fc007_88c3_4a18_8a21_b1cba09ed51c.slice/crio-9d37439e886bdb56dd88e6321e3cc76088ed1c13a9ebdfee1fd8ed7aca23300a WatchSource:0}: Error finding container 9d37439e886bdb56dd88e6321e3cc76088ed1c13a9ebdfee1fd8ed7aca23300a: Status 404 returned error can't find the container with id 9d37439e886bdb56dd88e6321e3cc76088ed1c13a9ebdfee1fd8ed7aca23300a
Sep 10 15:42:56.425510 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:56.425432    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw" event={"ID":"f66fc007-88c3-4a18-8a21-b1cba09ed51c","Type":"ContainerStarted","Data":"c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f"}
Sep 10 15:42:56.425985 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:56.425530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw" event={"ID":"f66fc007-88c3-4a18-8a21-b1cba09ed51c","Type":"ContainerStarted","Data":"9d37439e886bdb56dd88e6321e3cc76088ed1c13a9ebdfee1fd8ed7aca23300a"}
Sep 10 15:42:56.425985 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:56.425562    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:42:56.427134 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:56.427107    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj" event={"ID":"1788655a-cec8-4786-80e4-2963e93f7b6d","Type":"ContainerStarted","Data":"92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2"}
Sep 10 15:42:56.427274 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:56.427229    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:42:56.444605 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:56.444541    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw" podStartSLOduration=2.444519763 podStartE2EDuration="2.444519763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:56.444119385 +0000 UTC m=+1616.980074961" watchObservedRunningTime="2026-09-10 15:42:56.444519763 +0000 UTC m=+1616.980475326"
Sep 10 15:42:56.461429 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:42:56.461372    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj" podStartSLOduration=2.461355208 podStartE2EDuration="2.461355208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:56.460180957 +0000 UTC m=+1616.996136549" watchObservedRunningTime="2026-09-10 15:42:56.461355208 +0000 UTC m=+1616.997310769"
Sep 10 15:43:03.227663 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:03.227625    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:43:07.441658 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:07.441621    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:43:07.442088 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:07.441858    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:43:17.899435 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:17.899389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:18.516753 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.516715    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"]
Sep 10 15:43:18.520546 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.520527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"
Sep 10 15:43:18.523168 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.523142    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:43:18.523313 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.523142    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:43:18.528682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.528393    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"]
Sep 10 15:43:18.544275 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.544245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05577de1-3dd4-4374-9429-c12d0b86b463-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt\" (UID: \"05577de1-3dd4-4374-9429-c12d0b86b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"
Sep 10 15:43:18.644902 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.644857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05577de1-3dd4-4374-9429-c12d0b86b463-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt\" (UID: \"05577de1-3dd4-4374-9429-c12d0b86b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"
Sep 10 15:43:18.645546 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.645525    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05577de1-3dd4-4374-9429-c12d0b86b463-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt\" (UID: \"05577de1-3dd4-4374-9429-c12d0b86b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"
Sep 10 15:43:18.832975 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:18.832944    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"
Sep 10 15:43:19.108062 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:19.108032    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"]
Sep 10 15:43:19.109908 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:43:19.109884    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05577de1_3dd4_4374_9429_c12d0b86b463.slice/crio-5f9c25a6e7718d2884d64f5332f940cf4fb36443339e79e10f819fd11d5b494f WatchSource:0}: Error finding container 5f9c25a6e7718d2884d64f5332f940cf4fb36443339e79e10f819fd11d5b494f: Status 404 returned error can't find the container with id 5f9c25a6e7718d2884d64f5332f940cf4fb36443339e79e10f819fd11d5b494f
Sep 10 15:43:19.574069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:19.574023    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt" event={"ID":"05577de1-3dd4-4374-9429-c12d0b86b463","Type":"ContainerStarted","Data":"b19c55922dafa5746be81820ea9d10e91d7aa95699aefd0c32751661b779dcff"}
Sep 10 15:43:19.574257 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:19.574126    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt" event={"ID":"05577de1-3dd4-4374-9429-c12d0b86b463","Type":"ContainerStarted","Data":"5f9c25a6e7718d2884d64f5332f940cf4fb36443339e79e10f819fd11d5b494f"}
Sep 10 15:43:19.592548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:19.592492    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt" podStartSLOduration=1.592477122 podStartE2EDuration="1.592477122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:19.590605399 +0000 UTC m=+1640.126560970" watchObservedRunningTime="2026-09-10 15:43:19.592477122 +0000 UTC m=+1640.128432683"
Sep 10 15:43:20.580698 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:20.580662    2562 generic.go:356] "Generic (PLEG): container finished" podID="05577de1-3dd4-4374-9429-c12d0b86b463" containerID="b19c55922dafa5746be81820ea9d10e91d7aa95699aefd0c32751661b779dcff" exitCode=0
Sep 10 15:43:20.581112 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:20.580717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt" event={"ID":"05577de1-3dd4-4374-9429-c12d0b86b463","Type":"ContainerDied","Data":"b19c55922dafa5746be81820ea9d10e91d7aa95699aefd0c32751661b779dcff"}
Sep 10 15:43:21.733517 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:21.733487    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"
Sep 10 15:43:21.774045 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:21.774012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05577de1-3dd4-4374-9429-c12d0b86b463-maas-api-service-ca\") pod \"05577de1-3dd4-4374-9429-c12d0b86b463\" (UID: \"05577de1-3dd4-4374-9429-c12d0b86b463\") "
Sep 10 15:43:21.774443 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:21.774415    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05577de1-3dd4-4374-9429-c12d0b86b463-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05577de1-3dd4-4374-9429-c12d0b86b463" (UID: "05577de1-3dd4-4374-9429-c12d0b86b463"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:21.875214 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:21.875105    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05577de1-3dd4-4374-9429-c12d0b86b463-maas-api-service-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:22.595414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:22.595383    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"
Sep 10 15:43:22.595642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:22.595419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt" event={"ID":"05577de1-3dd4-4374-9429-c12d0b86b463","Type":"ContainerDied","Data":"5f9c25a6e7718d2884d64f5332f940cf4fb36443339e79e10f819fd11d5b494f"}
Sep 10 15:43:22.595642 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:22.595481    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f9c25a6e7718d2884d64f5332f940cf4fb36443339e79e10f819fd11d5b494f"
Sep 10 15:43:24.751485 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.751433    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"]
Sep 10 15:43:24.752013 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.751752    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw" podUID="f66fc007-88c3-4a18-8a21-b1cba09ed51c" containerName="payload-processing" containerID="cri-o://c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f" gracePeriod=30
Sep 10 15:43:24.759941 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.758771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:24.761571 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.761528    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"]
Sep 10 15:43:24.762263 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.761921    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj" podUID="1788655a-cec8-4786-80e4-2963e93f7b6d" containerName="payload-pre-processing" containerID="cri-o://92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2" gracePeriod=30
Sep 10 15:43:24.868278 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.868241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:24.894247 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.894214    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"]
Sep 10 15:43:24.896430 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.896396    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-810029-49mwt"]
Sep 10 15:43:24.966777 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:24.966726    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:25.044038 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.044015    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:43:25.047305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.047282    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:43:25.106587 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.106557    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsm7c\" (UniqueName: \"kubernetes.io/projected/f66fc007-88c3-4a18-8a21-b1cba09ed51c-kube-api-access-wsm7c\") pod \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") "
Sep 10 15:43:25.106765 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.106596    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrw8k\" (UniqueName: \"kubernetes.io/projected/1788655a-cec8-4786-80e4-2963e93f7b6d-kube-api-access-vrw8k\") pod \"1788655a-cec8-4786-80e4-2963e93f7b6d\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") "
Sep 10 15:43:25.106765 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.106621    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66fc007-88c3-4a18-8a21-b1cba09ed51c-plugins-config-volume\") pod \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\" (UID: \"f66fc007-88c3-4a18-8a21-b1cba09ed51c\") "
Sep 10 15:43:25.106765 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.106727    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1788655a-cec8-4786-80e4-2963e93f7b6d-plugins-config-volume\") pod \"1788655a-cec8-4786-80e4-2963e93f7b6d\" (UID: \"1788655a-cec8-4786-80e4-2963e93f7b6d\") "
Sep 10 15:43:25.107105 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.107075    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66fc007-88c3-4a18-8a21-b1cba09ed51c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f66fc007-88c3-4a18-8a21-b1cba09ed51c" (UID: "f66fc007-88c3-4a18-8a21-b1cba09ed51c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:25.107235 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.107154    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1788655a-cec8-4786-80e4-2963e93f7b6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1788655a-cec8-4786-80e4-2963e93f7b6d" (UID: "1788655a-cec8-4786-80e4-2963e93f7b6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:25.108829 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.108808    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1788655a-cec8-4786-80e4-2963e93f7b6d-kube-api-access-vrw8k" (OuterVolumeSpecName: "kube-api-access-vrw8k") pod "1788655a-cec8-4786-80e4-2963e93f7b6d" (UID: "1788655a-cec8-4786-80e4-2963e93f7b6d"). InnerVolumeSpecName "kube-api-access-vrw8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:25.109385 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.109361    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66fc007-88c3-4a18-8a21-b1cba09ed51c-kube-api-access-wsm7c" (OuterVolumeSpecName: "kube-api-access-wsm7c") pod "f66fc007-88c3-4a18-8a21-b1cba09ed51c" (UID: "f66fc007-88c3-4a18-8a21-b1cba09ed51c"). InnerVolumeSpecName "kube-api-access-wsm7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:25.208037 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.207998    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsm7c\" (UniqueName: \"kubernetes.io/projected/f66fc007-88c3-4a18-8a21-b1cba09ed51c-kube-api-access-wsm7c\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:25.208037 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.208041    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrw8k\" (UniqueName: \"kubernetes.io/projected/1788655a-cec8-4786-80e4-2963e93f7b6d-kube-api-access-vrw8k\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:25.208241 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.208051    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66fc007-88c3-4a18-8a21-b1cba09ed51c-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:25.208241 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.208061    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1788655a-cec8-4786-80e4-2963e93f7b6d-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:25.257356 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.257323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:25.611175 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.611140    2562 generic.go:356] "Generic (PLEG): container finished" podID="1788655a-cec8-4786-80e4-2963e93f7b6d" containerID="92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2" exitCode=0
Sep 10 15:43:25.611376 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.611190    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"
Sep 10 15:43:25.611376 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.611231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj" event={"ID":"1788655a-cec8-4786-80e4-2963e93f7b6d","Type":"ContainerDied","Data":"92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2"}
Sep 10 15:43:25.611376 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.611298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj" event={"ID":"1788655a-cec8-4786-80e4-2963e93f7b6d","Type":"ContainerDied","Data":"533d1212b4e76e9f6d23476807a29e9d1a012bd8f8e34cbc269f318cca8e813b"}
Sep 10 15:43:25.611376 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.611331    2562 scope.go:117] "RemoveContainer" containerID="92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2"
Sep 10 15:43:25.612884 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.612861    2562 generic.go:356] "Generic (PLEG): container finished" podID="f66fc007-88c3-4a18-8a21-b1cba09ed51c" containerID="c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f" exitCode=0
Sep 10 15:43:25.613002 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.612906    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw" event={"ID":"f66fc007-88c3-4a18-8a21-b1cba09ed51c","Type":"ContainerDied","Data":"c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f"}
Sep 10 15:43:25.613002 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.612919    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"
Sep 10 15:43:25.613002 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.612945    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw" event={"ID":"f66fc007-88c3-4a18-8a21-b1cba09ed51c","Type":"ContainerDied","Data":"9d37439e886bdb56dd88e6321e3cc76088ed1c13a9ebdfee1fd8ed7aca23300a"}
Sep 10 15:43:25.621779 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.621754    2562 scope.go:117] "RemoveContainer" containerID="92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2"
Sep 10 15:43:25.622130 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:43:25.622108    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2\": container with ID starting with 92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2 not found: ID does not exist" containerID="92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2"
Sep 10 15:43:25.622228 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.622139    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2"} err="failed to get container status \"92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2\": rpc error: code = NotFound desc = could not find container \"92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2\": container with ID starting with 92ae601b575b22af927ba641174d5f8fdd13e24f673d7534caa931cb33937ab2 not found: ID does not exist"
Sep 10 15:43:25.622228 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.622157    2562 scope.go:117] "RemoveContainer" containerID="c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f"
Sep 10 15:43:25.631013 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.630993    2562 scope.go:117] "RemoveContainer" containerID="c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f"
Sep 10 15:43:25.631297 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:43:25.631280    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f\": container with ID starting with c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f not found: ID does not exist" containerID="c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f"
Sep 10 15:43:25.631351 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.631303    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f"} err="failed to get container status \"c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f\": rpc error: code = NotFound desc = could not find container \"c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f\": container with ID starting with c6a5c3f28eb0381acb81a9b0f0378852c11dca9ae9bc8b6e22fb3b6c7e57af8f not found: ID does not exist"
Sep 10 15:43:25.635286 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.635260    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"]
Sep 10 15:43:25.639234 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.639211    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-810029-ddb9b97f4-7wbjj"]
Sep 10 15:43:25.649194 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.649163    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"]
Sep 10 15:43:25.655438 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.655407    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-810029-9fcfcdcc5-c48sw"]
Sep 10 15:43:25.955385 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.955294    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05577de1-3dd4-4374-9429-c12d0b86b463" path="/var/lib/kubelet/pods/05577de1-3dd4-4374-9429-c12d0b86b463/volumes"
Sep 10 15:43:25.955894 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.955870    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1788655a-cec8-4786-80e4-2963e93f7b6d" path="/var/lib/kubelet/pods/1788655a-cec8-4786-80e4-2963e93f7b6d/volumes"
Sep 10 15:43:25.956286 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:25.956267    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f66fc007-88c3-4a18-8a21-b1cba09ed51c" path="/var/lib/kubelet/pods/f66fc007-88c3-4a18-8a21-b1cba09ed51c/volumes"
Sep 10 15:43:29.319518 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:29.319481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:43:29.687993 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:29.687908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:31.910466 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:31.910426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:32.055575 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:32.055537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:32.164742 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:32.164655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:35.610761 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.610724    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"]
Sep 10 15:43:35.611223 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611208    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1788655a-cec8-4786-80e4-2963e93f7b6d" containerName="payload-pre-processing"
Sep 10 15:43:35.611223 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611224    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1788655a-cec8-4786-80e4-2963e93f7b6d" containerName="payload-pre-processing"
Sep 10 15:43:35.611310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611236    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05577de1-3dd4-4374-9429-c12d0b86b463" containerName="revoke-keys"
Sep 10 15:43:35.611310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611242    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="05577de1-3dd4-4374-9429-c12d0b86b463" containerName="revoke-keys"
Sep 10 15:43:35.611310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611259    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f66fc007-88c3-4a18-8a21-b1cba09ed51c" containerName="payload-processing"
Sep 10 15:43:35.611310 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611265    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f66fc007-88c3-4a18-8a21-b1cba09ed51c" containerName="payload-processing"
Sep 10 15:43:35.611437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611324    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1788655a-cec8-4786-80e4-2963e93f7b6d" containerName="payload-pre-processing"
Sep 10 15:43:35.611437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611336    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f66fc007-88c3-4a18-8a21-b1cba09ed51c" containerName="payload-processing"
Sep 10 15:43:35.611437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.611343    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="05577de1-3dd4-4374-9429-c12d0b86b463" containerName="revoke-keys"
Sep 10 15:43:35.614399 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.614381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"
Sep 10 15:43:35.616776 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.616749    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:43:35.616897 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.616825    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:43:35.625099 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.625074    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"]
Sep 10 15:43:35.715422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.715388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287c992-2be6-478d-bfd6-34cb4424b884-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx\" (UID: \"1287c992-2be6-478d-bfd6-34cb4424b884\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"
Sep 10 15:43:35.776857 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.776818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:35.816802 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.816765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287c992-2be6-478d-bfd6-34cb4424b884-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx\" (UID: \"1287c992-2be6-478d-bfd6-34cb4424b884\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"
Sep 10 15:43:35.817386 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.817368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287c992-2be6-478d-bfd6-34cb4424b884-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx\" (UID: \"1287c992-2be6-478d-bfd6-34cb4424b884\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"
Sep 10 15:43:35.924947 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:35.924869    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"
Sep 10 15:43:36.194422 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:36.194250    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"]
Sep 10 15:43:36.196742 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:43:36.196716    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1287c992_2be6_478d_bfd6_34cb4424b884.slice/crio-e5f66a7b720bfd829febb32d6e8df73e06ada54cb83e7102b9c114eea4adde2c WatchSource:0}: Error finding container e5f66a7b720bfd829febb32d6e8df73e06ada54cb83e7102b9c114eea4adde2c: Status 404 returned error can't find the container with id e5f66a7b720bfd829febb32d6e8df73e06ada54cb83e7102b9c114eea4adde2c
Sep 10 15:43:36.673255 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:36.673216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx" event={"ID":"1287c992-2be6-478d-bfd6-34cb4424b884","Type":"ContainerStarted","Data":"798bec52074afae9501c38824d96f157c72a1bfa1d64ea1a964ef1964c9174db"}
Sep 10 15:43:36.673727 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:36.673266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx" event={"ID":"1287c992-2be6-478d-bfd6-34cb4424b884","Type":"ContainerStarted","Data":"e5f66a7b720bfd829febb32d6e8df73e06ada54cb83e7102b9c114eea4adde2c"}
Sep 10 15:43:36.693841 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:36.693789    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx" podStartSLOduration=1.693771342 podStartE2EDuration="1.693771342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:43:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:36.692546385 +0000 UTC m=+1657.228501968" watchObservedRunningTime="2026-09-10 15:43:36.693771342 +0000 UTC m=+1657.229726943"
Sep 10 15:43:37.680034 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:37.679998    2562 generic.go:356] "Generic (PLEG): container finished" podID="1287c992-2be6-478d-bfd6-34cb4424b884" containerID="798bec52074afae9501c38824d96f157c72a1bfa1d64ea1a964ef1964c9174db" exitCode=0
Sep 10 15:43:37.680034 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:37.680037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx" event={"ID":"1287c992-2be6-478d-bfd6-34cb4424b884","Type":"ContainerDied","Data":"798bec52074afae9501c38824d96f157c72a1bfa1d64ea1a964ef1964c9174db"}
Sep 10 15:43:38.819256 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:38.819230    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"
Sep 10 15:43:38.950406 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:38.950373    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287c992-2be6-478d-bfd6-34cb4424b884-maas-api-service-ca\") pod \"1287c992-2be6-478d-bfd6-34cb4424b884\" (UID: \"1287c992-2be6-478d-bfd6-34cb4424b884\") "
Sep 10 15:43:38.950761 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:38.950736    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1287c992-2be6-478d-bfd6-34cb4424b884-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1287c992-2be6-478d-bfd6-34cb4424b884" (UID: "1287c992-2be6-478d-bfd6-34cb4424b884"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:39.051481 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:39.051360    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287c992-2be6-478d-bfd6-34cb4424b884-maas-api-service-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:39.692421 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:39.692389    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"
Sep 10 15:43:39.692621 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:39.692392    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx" event={"ID":"1287c992-2be6-478d-bfd6-34cb4424b884","Type":"ContainerDied","Data":"e5f66a7b720bfd829febb32d6e8df73e06ada54cb83e7102b9c114eea4adde2c"}
Sep 10 15:43:39.692621 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:39.692507    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5f66a7b720bfd829febb32d6e8df73e06ada54cb83e7102b9c114eea4adde2c"
Sep 10 15:43:49.125240 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:49.125198    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:43:51.086433 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.086393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:51.093089 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.093050    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"]
Sep 10 15:43:51.093357 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.093326    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf" podUID="e09804b5-64d8-45b8-9ede-6dc1a0f7a791" containerName="payload-pre-processing" containerID="cri-o://618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa" gracePeriod=30
Sep 10 15:43:51.183041 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.183004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:51.235629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.235590    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"]
Sep 10 15:43:51.237628 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.237601    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ba5c7e-fmsmx"]
Sep 10 15:43:51.258963 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.258932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:51.343370 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.343310    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:43:51.380444 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.380401    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tzx4\" (UniqueName: \"kubernetes.io/projected/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-kube-api-access-4tzx4\") pod \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") "
Sep 10 15:43:51.380648 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.380568    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-plugins-config-volume\") pod \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\" (UID: \"e09804b5-64d8-45b8-9ede-6dc1a0f7a791\") "
Sep 10 15:43:51.380948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.380922    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e09804b5-64d8-45b8-9ede-6dc1a0f7a791" (UID: "e09804b5-64d8-45b8-9ede-6dc1a0f7a791"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:51.382699 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.382674    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-kube-api-access-4tzx4" (OuterVolumeSpecName: "kube-api-access-4tzx4") pod "e09804b5-64d8-45b8-9ede-6dc1a0f7a791" (UID: "e09804b5-64d8-45b8-9ede-6dc1a0f7a791"). InnerVolumeSpecName "kube-api-access-4tzx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:51.481355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.481317    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:51.481355 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.481346    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tzx4\" (UniqueName: \"kubernetes.io/projected/e09804b5-64d8-45b8-9ede-6dc1a0f7a791-kube-api-access-4tzx4\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:43:51.611629 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.611539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:51.759272 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.759236    2562 generic.go:356] "Generic (PLEG): container finished" podID="e09804b5-64d8-45b8-9ede-6dc1a0f7a791" containerID="618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa" exitCode=0
Sep 10 15:43:51.759489 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.759282    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"
Sep 10 15:43:51.759489 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.759326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf" event={"ID":"e09804b5-64d8-45b8-9ede-6dc1a0f7a791","Type":"ContainerDied","Data":"618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa"}
Sep 10 15:43:51.759489 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.759382    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf" event={"ID":"e09804b5-64d8-45b8-9ede-6dc1a0f7a791","Type":"ContainerDied","Data":"36a7322deb5e349af061543ec3ff65626f35ff39722052e50225ae33005e7004"}
Sep 10 15:43:51.759489 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.759422    2562 scope.go:117] "RemoveContainer" containerID="618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa"
Sep 10 15:43:51.769251 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.769236    2562 scope.go:117] "RemoveContainer" containerID="618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa"
Sep 10 15:43:51.769574 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:43:51.769557    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa\": container with ID starting with 618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa not found: ID does not exist" containerID="618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa"
Sep 10 15:43:51.769652 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.769580    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa"} err="failed to get container status \"618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa\": rpc error: code = NotFound desc = could not find container \"618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa\": container with ID starting with 618138873b783c60909c09c787220f45af87e68bf5b2a22b5ab5016fa33759fa not found: ID does not exist"
Sep 10 15:43:51.780294 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.780271    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"]
Sep 10 15:43:51.783276 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.783253    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ba5c7e-7c6cb679cd-c7cvf"]
Sep 10 15:43:51.960149 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.960067    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1287c992-2be6-478d-bfd6-34cb4424b884" path="/var/lib/kubelet/pods/1287c992-2be6-478d-bfd6-34cb4424b884/volumes"
Sep 10 15:43:51.960781 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:51.960761    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e09804b5-64d8-45b8-9ede-6dc1a0f7a791" path="/var/lib/kubelet/pods/e09804b5-64d8-45b8-9ede-6dc1a0f7a791/volumes"
Sep 10 15:43:56.628417 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:56.628381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:58.069808 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:43:58.069772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:01.216048 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:01.216006    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:44:02.980129 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:02.980092    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:33.542822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:33.542731    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:34.007373 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:34.007341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:39.850219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:39.850177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:39.953222 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:39.953182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:40.139971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.139881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:40.480360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.480279    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"]
Sep 10 15:44:40.480855 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.480834    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1287c992-2be6-478d-bfd6-34cb4424b884" containerName="revoke-keys"
Sep 10 15:44:40.480913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.480861    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1287c992-2be6-478d-bfd6-34cb4424b884" containerName="revoke-keys"
Sep 10 15:44:40.480913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.480882    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09804b5-64d8-45b8-9ede-6dc1a0f7a791" containerName="payload-pre-processing"
Sep 10 15:44:40.480913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.480891    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09804b5-64d8-45b8-9ede-6dc1a0f7a791" containerName="payload-pre-processing"
Sep 10 15:44:40.481019 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.481009    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1287c992-2be6-478d-bfd6-34cb4424b884" containerName="revoke-keys"
Sep 10 15:44:40.481055 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.481022    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e09804b5-64d8-45b8-9ede-6dc1a0f7a791" containerName="payload-pre-processing"
Sep 10 15:44:40.484308 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.484293    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"
Sep 10 15:44:40.486548 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.486527    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:44:40.486640 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.486529    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:44:40.490800 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.490771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"]
Sep 10 15:44:40.572199 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.572162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4cbe4a-7069-474f-baea-bdc92b27dd3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw\" (UID: \"ab4cbe4a-7069-474f-baea-bdc92b27dd3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"
Sep 10 15:44:40.581328 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.581292    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:40.672992 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.672955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4cbe4a-7069-474f-baea-bdc92b27dd3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw\" (UID: \"ab4cbe4a-7069-474f-baea-bdc92b27dd3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"
Sep 10 15:44:40.673813 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.673790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4cbe4a-7069-474f-baea-bdc92b27dd3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw\" (UID: \"ab4cbe4a-7069-474f-baea-bdc92b27dd3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"
Sep 10 15:44:40.797767 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:40.797672    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"
Sep 10 15:44:41.068737 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:41.068712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"]
Sep 10 15:44:41.070733 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:44:41.070702    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4cbe4a_7069_474f_baea_bdc92b27dd3c.slice/crio-313ca51b4e6e74213a1d672bf224f88341739b9ae486c795927582d7be320fdd WatchSource:0}: Error finding container 313ca51b4e6e74213a1d672bf224f88341739b9ae486c795927582d7be320fdd: Status 404 returned error can't find the container with id 313ca51b4e6e74213a1d672bf224f88341739b9ae486c795927582d7be320fdd
Sep 10 15:44:42.035666 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:42.035617    2562 generic.go:356] "Generic (PLEG): container finished" podID="ab4cbe4a-7069-474f-baea-bdc92b27dd3c" containerID="05903a2dd5def2477c035d3969cb5f14344e5a2f27efe7e932a59995d7eee20f" exitCode=0
Sep 10 15:44:42.035839 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:42.035679    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw" event={"ID":"ab4cbe4a-7069-474f-baea-bdc92b27dd3c","Type":"ContainerDied","Data":"05903a2dd5def2477c035d3969cb5f14344e5a2f27efe7e932a59995d7eee20f"}
Sep 10 15:44:42.035839 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:42.035727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw" event={"ID":"ab4cbe4a-7069-474f-baea-bdc92b27dd3c","Type":"ContainerStarted","Data":"313ca51b4e6e74213a1d672bf224f88341739b9ae486c795927582d7be320fdd"}
Sep 10 15:44:43.185970 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:43.185943    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"
Sep 10 15:44:43.300881 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:43.300795    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4cbe4a-7069-474f-baea-bdc92b27dd3c-maas-api-service-ca\") pod \"ab4cbe4a-7069-474f-baea-bdc92b27dd3c\" (UID: \"ab4cbe4a-7069-474f-baea-bdc92b27dd3c\") "
Sep 10 15:44:43.301144 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:43.301122    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab4cbe4a-7069-474f-baea-bdc92b27dd3c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab4cbe4a-7069-474f-baea-bdc92b27dd3c" (UID: "ab4cbe4a-7069-474f-baea-bdc92b27dd3c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:44:43.401776 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:43.401739    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab4cbe4a-7069-474f-baea-bdc92b27dd3c-maas-api-service-ca\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:44:44.046856 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:44.046826    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"
Sep 10 15:44:44.047035 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:44.046861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw" event={"ID":"ab4cbe4a-7069-474f-baea-bdc92b27dd3c","Type":"ContainerDied","Data":"313ca51b4e6e74213a1d672bf224f88341739b9ae486c795927582d7be320fdd"}
Sep 10 15:44:44.047035 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:44.046910    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="313ca51b4e6e74213a1d672bf224f88341739b9ae486c795927582d7be320fdd"
Sep 10 15:44:51.071923 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.071878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:51.096133 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.096095    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"]
Sep 10 15:44:51.096316 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.096296    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb" podUID="ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" containerName="payload-processing" containerID="cri-o://9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d" gracePeriod=30
Sep 10 15:44:51.172053 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.172021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:51.237765 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.237724    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"]
Sep 10 15:44:51.240103 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.240079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:51.241552 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.241525    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-09b833-q7llw"]
Sep 10 15:44:51.299902 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.299870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:51.361539 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.361514    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:44:51.486597 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.486561    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhc2n\" (UniqueName: \"kubernetes.io/projected/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-kube-api-access-bhc2n\") pod \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") "
Sep 10 15:44:51.486769 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.486669    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-plugins-config-volume\") pod \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\" (UID: \"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5\") "
Sep 10 15:44:51.487075 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.487051    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" (UID: "ed91e65c-200f-4a9f-98bc-5edd3f45ddb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:44:51.488790 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.488770    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-kube-api-access-bhc2n" (OuterVolumeSpecName: "kube-api-access-bhc2n") pod "ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" (UID: "ed91e65c-200f-4a9f-98bc-5edd3f45ddb5"). InnerVolumeSpecName "kube-api-access-bhc2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:44:51.588281 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.588240    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhc2n\" (UniqueName: \"kubernetes.io/projected/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-kube-api-access-bhc2n\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:44:51.588281 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.588273    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:44:51.618178 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.618096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:51.961476 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:51.957124    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab4cbe4a-7069-474f-baea-bdc92b27dd3c" path="/var/lib/kubelet/pods/ab4cbe4a-7069-474f-baea-bdc92b27dd3c/volumes"
Sep 10 15:44:52.094032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.093998    2562 generic.go:356] "Generic (PLEG): container finished" podID="ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" containerID="9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d" exitCode=0
Sep 10 15:44:52.094032 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.094036    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb" event={"ID":"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5","Type":"ContainerDied","Data":"9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d"}
Sep 10 15:44:52.094543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.094050    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"
Sep 10 15:44:52.094543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.094081    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb" event={"ID":"ed91e65c-200f-4a9f-98bc-5edd3f45ddb5","Type":"ContainerDied","Data":"cfc3498b2f59639cf73ae3f29c68a9f2d295a11bb44d6622135817ee22b8eb88"}
Sep 10 15:44:52.094543 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.094104    2562 scope.go:117] "RemoveContainer" containerID="9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d"
Sep 10 15:44:52.103537 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.103364    2562 scope.go:117] "RemoveContainer" containerID="9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d"
Sep 10 15:44:52.103659 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:44:52.103640    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d\": container with ID starting with 9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d not found: ID does not exist" containerID="9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d"
Sep 10 15:44:52.103704 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.103671    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d"} err="failed to get container status \"9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d\": rpc error: code = NotFound desc = could not find container \"9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d\": container with ID starting with 9e1beb639336f31834ee9abed349967be89a1143eb89ef33e7e2a5c7fcbebf0d not found: ID does not exist"
Sep 10 15:44:52.109051 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.109027    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"]
Sep 10 15:44:52.112817 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:52.112794    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-09b833-5db4658765-bcptb"]
Sep 10 15:44:53.955342 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:53.955311    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" path="/var/lib/kubelet/pods/ed91e65c-200f-4a9f-98bc-5edd3f45ddb5/volumes"
Sep 10 15:44:54.317804 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:54.317764    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:44:57.815931 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:57.815892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:57.918153 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:57.918121    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:58.073013 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:58.072922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:58.552022 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:44:58.551935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:00.129494 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.129446    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z"]
Sep 10 15:45:00.129900 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.129886    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" containerName="payload-processing"
Sep 10 15:45:00.129952 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.129901    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" containerName="payload-processing"
Sep 10 15:45:00.129952 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.129919    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab4cbe4a-7069-474f-baea-bdc92b27dd3c" containerName="revoke-keys"
Sep 10 15:45:00.129952 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.129925    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab4cbe4a-7069-474f-baea-bdc92b27dd3c" containerName="revoke-keys"
Sep 10 15:45:00.130047 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.129984    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed91e65c-200f-4a9f-98bc-5edd3f45ddb5" containerName="payload-processing"
Sep 10 15:45:00.130047 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.129994    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab4cbe4a-7069-474f-baea-bdc92b27dd3c" containerName="revoke-keys"
Sep 10 15:45:00.134895 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.134868    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"]
Sep 10 15:45:00.135086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.134987    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z"
Sep 10 15:45:00.137443 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.137412    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zzl2g\""
Sep 10 15:45:00.138792 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.138777    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"
Sep 10 15:45:00.142414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.142386    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z"]
Sep 10 15:45:00.145918 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.145888    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"]
Sep 10 15:45:00.150803 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.150619    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z"
Sep 10 15:45:00.157293 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.157275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"
Sep 10 15:45:00.442634 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.442604    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"]
Sep 10 15:45:00.444345 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:45:00.444314    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae033d1_1ada_4943_9660_966b62e423a1.slice/crio-25a951835048714dc529ac86de4c4688076a02942a7df9c7496d3e97a646b435 WatchSource:0}: Error finding container 25a951835048714dc529ac86de4c4688076a02942a7df9c7496d3e97a646b435: Status 404 returned error can't find the container with id 25a951835048714dc529ac86de4c4688076a02942a7df9c7496d3e97a646b435
Sep 10 15:45:00.455686 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:00.455659    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z"]
Sep 10 15:45:00.458965 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:45:00.458943    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5965974_57b1_45b7_8f2b_c3a732b256cc.slice/crio-18cc76d3aad35bed84aac6bbb7ffbcc2bf7729dbd69ae1ec42c60e51c52d141b WatchSource:0}: Error finding container 18cc76d3aad35bed84aac6bbb7ffbcc2bf7729dbd69ae1ec42c60e51c52d141b: Status 404 returned error can't find the container with id 18cc76d3aad35bed84aac6bbb7ffbcc2bf7729dbd69ae1ec42c60e51c52d141b
Sep 10 15:45:01.145810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:01.145771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp" event={"ID":"9ae033d1-1ada-4943-9660-966b62e423a1","Type":"ContainerStarted","Data":"80bceab56b1fdf292206b3364d89a0ffcefff17236e96aa953d997c35a5b0f22"}
Sep 10 15:45:01.146239 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:01.145829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp" event={"ID":"9ae033d1-1ada-4943-9660-966b62e423a1","Type":"ContainerStarted","Data":"25a951835048714dc529ac86de4c4688076a02942a7df9c7496d3e97a646b435"}
Sep 10 15:45:01.147564 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:01.147537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z" event={"ID":"f5965974-57b1-45b7-8f2b-c3a732b256cc","Type":"ContainerStarted","Data":"901059a20d16393805eafbd706d10def1127e365c96d2e0780f5caf2947b9a4a"}
Sep 10 15:45:01.147681 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:01.147587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z" event={"ID":"f5965974-57b1-45b7-8f2b-c3a732b256cc","Type":"ContainerStarted","Data":"18cc76d3aad35bed84aac6bbb7ffbcc2bf7729dbd69ae1ec42c60e51c52d141b"}
Sep 10 15:45:01.160196 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:01.160149    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp" podStartSLOduration=1.160134017 podStartE2EDuration="1.160134017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:45:01.158879334 +0000 UTC m=+1741.694834897" watchObservedRunningTime="2026-09-10 15:45:01.160134017 +0000 UTC m=+1741.696089579"
Sep 10 15:45:01.172957 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:01.172906    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z" podStartSLOduration=1.172893501 podStartE2EDuration="1.172893501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:45:01.171851898 +0000 UTC m=+1741.707807494" watchObservedRunningTime="2026-09-10 15:45:01.172893501 +0000 UTC m=+1741.708849063"
Sep 10 15:45:02.154447 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:02.154410    2562 generic.go:356] "Generic (PLEG): container finished" podID="9ae033d1-1ada-4943-9660-966b62e423a1" containerID="80bceab56b1fdf292206b3364d89a0ffcefff17236e96aa953d997c35a5b0f22" exitCode=0
Sep 10 15:45:02.154872 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:02.154493    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp" event={"ID":"9ae033d1-1ada-4943-9660-966b62e423a1","Type":"ContainerDied","Data":"80bceab56b1fdf292206b3364d89a0ffcefff17236e96aa953d997c35a5b0f22"}
Sep 10 15:45:02.156101 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:02.156079    2562 generic.go:356] "Generic (PLEG): container finished" podID="f5965974-57b1-45b7-8f2b-c3a732b256cc" containerID="901059a20d16393805eafbd706d10def1127e365c96d2e0780f5caf2947b9a4a" exitCode=0
Sep 10 15:45:02.156198 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:02.156120    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z" event={"ID":"f5965974-57b1-45b7-8f2b-c3a732b256cc","Type":"ContainerDied","Data":"901059a20d16393805eafbd706d10def1127e365c96d2e0780f5caf2947b9a4a"}
Sep 10 15:45:03.311292 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:03.311264    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"
Sep 10 15:45:03.314526 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:03.314503    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z"
Sep 10 15:45:04.167171 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:04.167141    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"
Sep 10 15:45:04.167352 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:04.167141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp" event={"ID":"9ae033d1-1ada-4943-9660-966b62e423a1","Type":"ContainerDied","Data":"25a951835048714dc529ac86de4c4688076a02942a7df9c7496d3e97a646b435"}
Sep 10 15:45:04.167352 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:04.167261    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25a951835048714dc529ac86de4c4688076a02942a7df9c7496d3e97a646b435"
Sep 10 15:45:04.168615 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:04.168597    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z"
Sep 10 15:45:04.168710 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:04.168636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817585-pmh2z" event={"ID":"f5965974-57b1-45b7-8f2b-c3a732b256cc","Type":"ContainerDied","Data":"18cc76d3aad35bed84aac6bbb7ffbcc2bf7729dbd69ae1ec42c60e51c52d141b"}
Sep 10 15:45:04.168710 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:04.168677    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18cc76d3aad35bed84aac6bbb7ffbcc2bf7729dbd69ae1ec42c60e51c52d141b"
Sep 10 15:45:08.865293 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:08.865252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:08.869925 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:08.869890    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"]
Sep 10 15:45:08.870188 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:08.870141    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5" podUID="1ba3a57f-92e5-4de4-acc1-277d1574bffe" containerName="payload-pre-processing" containerID="cri-o://d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7" gracePeriod=30
Sep 10 15:45:08.973728 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:08.973693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:09.067111 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.067074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:09.134069 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.134040    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:45:09.152591 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.152555    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfllq\" (UniqueName: \"kubernetes.io/projected/1ba3a57f-92e5-4de4-acc1-277d1574bffe-kube-api-access-lfllq\") pod \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") "
Sep 10 15:45:09.152788 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.152760    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ba3a57f-92e5-4de4-acc1-277d1574bffe-plugins-config-volume\") pod \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\" (UID: \"1ba3a57f-92e5-4de4-acc1-277d1574bffe\") "
Sep 10 15:45:09.153225 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.153184    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ba3a57f-92e5-4de4-acc1-277d1574bffe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ba3a57f-92e5-4de4-acc1-277d1574bffe" (UID: "1ba3a57f-92e5-4de4-acc1-277d1574bffe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:45:09.154913 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.154888    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ba3a57f-92e5-4de4-acc1-277d1574bffe-kube-api-access-lfllq" (OuterVolumeSpecName: "kube-api-access-lfllq") pod "1ba3a57f-92e5-4de4-acc1-277d1574bffe" (UID: "1ba3a57f-92e5-4de4-acc1-277d1574bffe"). InnerVolumeSpecName "kube-api-access-lfllq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:45:09.199256 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.199216    2562 generic.go:356] "Generic (PLEG): container finished" podID="1ba3a57f-92e5-4de4-acc1-277d1574bffe" containerID="d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7" exitCode=0
Sep 10 15:45:09.199488 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.199270    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"
Sep 10 15:45:09.199488 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.199305    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5" event={"ID":"1ba3a57f-92e5-4de4-acc1-277d1574bffe","Type":"ContainerDied","Data":"d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7"}
Sep 10 15:45:09.199488 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.199357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5" event={"ID":"1ba3a57f-92e5-4de4-acc1-277d1574bffe","Type":"ContainerDied","Data":"c4ffbf61bf00b2de6a98482351d4832e2d46822237b6b9291883ecee131c7f56"}
Sep 10 15:45:09.199488 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.199379    2562 scope.go:117] "RemoveContainer" containerID="d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7"
Sep 10 15:45:09.211015 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.210979    2562 scope.go:117] "RemoveContainer" containerID="d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7"
Sep 10 15:45:09.211603 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:45:09.211582    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7\": container with ID starting with d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7 not found: ID does not exist" containerID="d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7"
Sep 10 15:45:09.211693 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.211610    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7"} err="failed to get container status \"d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7\": rpc error: code = NotFound desc = could not find container \"d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7\": container with ID starting with d513ffb4017c39c36c0f92cf0b0eaa5ca588da3b80dfe2f95d33d78c1077d5a7 not found: ID does not exist"
Sep 10 15:45:09.220578 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.220550    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"]
Sep 10 15:45:09.225797 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.225770    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-62d2f7-f9fff4d78-x9qx5"]
Sep 10 15:45:09.254234 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.254196    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfllq\" (UniqueName: \"kubernetes.io/projected/1ba3a57f-92e5-4de4-acc1-277d1574bffe-kube-api-access-lfllq\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:45:09.254234 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.254228    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ba3a57f-92e5-4de4-acc1-277d1574bffe-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:45:09.394221 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.394136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:09.956931 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:09.956897    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ba3a57f-92e5-4de4-acc1-277d1574bffe" path="/var/lib/kubelet/pods/1ba3a57f-92e5-4de4-acc1-277d1574bffe/volumes"
Sep 10 15:45:11.919604 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:11.919561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:45:15.350594 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:15.350561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:15.450659 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:15.450616    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:15.550544 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:15.550510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:15.618095 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:15.618008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:16.339049 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:16.339012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:26.531691 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.531655    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"]
Sep 10 15:45:26.533519 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.533496    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc3e54-29817585-m9wmp"]
Sep 10 15:45:26.539683 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.539656    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:26.555282 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.555245    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"]
Sep 10 15:45:26.555532 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.555486    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh" podUID="d79cd1fc-5112-4e33-83a0-c46f038c485a" containerName="payload-processing" containerID="cri-o://5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800" gracePeriod=30
Sep 10 15:45:26.563305 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.563273    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"]
Sep 10 15:45:26.563579 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.563549    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j" podUID="57260c99-a3a6-4389-a4f7-bf1dd669d77d" containerName="payload-pre-processing" containerID="cri-o://655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9" gracePeriod=30
Sep 10 15:45:26.602286 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.602244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:26.704029 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.703992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:26.775960 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.775927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:26.813192 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.813166    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:45:26.844844 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.844810    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:45:26.933314 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.933265    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69n7f\" (UniqueName: \"kubernetes.io/projected/d79cd1fc-5112-4e33-83a0-c46f038c485a-kube-api-access-69n7f\") pod \"d79cd1fc-5112-4e33-83a0-c46f038c485a\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") "
Sep 10 15:45:26.933314 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.933316    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79cd1fc-5112-4e33-83a0-c46f038c485a-plugins-config-volume\") pod \"d79cd1fc-5112-4e33-83a0-c46f038c485a\" (UID: \"d79cd1fc-5112-4e33-83a0-c46f038c485a\") "
Sep 10 15:45:26.933575 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.933496    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wpwf\" (UniqueName: \"kubernetes.io/projected/57260c99-a3a6-4389-a4f7-bf1dd669d77d-kube-api-access-5wpwf\") pod \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") "
Sep 10 15:45:26.933632 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.933582    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57260c99-a3a6-4389-a4f7-bf1dd669d77d-plugins-config-volume\") pod \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\" (UID: \"57260c99-a3a6-4389-a4f7-bf1dd669d77d\") "
Sep 10 15:45:26.933695 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.933675    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d79cd1fc-5112-4e33-83a0-c46f038c485a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d79cd1fc-5112-4e33-83a0-c46f038c485a" (UID: "d79cd1fc-5112-4e33-83a0-c46f038c485a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:45:26.933954 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.933926    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57260c99-a3a6-4389-a4f7-bf1dd669d77d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "57260c99-a3a6-4389-a4f7-bf1dd669d77d" (UID: "57260c99-a3a6-4389-a4f7-bf1dd669d77d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:45:26.934065 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.933987    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79cd1fc-5112-4e33-83a0-c46f038c485a-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:45:26.935535 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.935512    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57260c99-a3a6-4389-a4f7-bf1dd669d77d-kube-api-access-5wpwf" (OuterVolumeSpecName: "kube-api-access-5wpwf") pod "57260c99-a3a6-4389-a4f7-bf1dd669d77d" (UID: "57260c99-a3a6-4389-a4f7-bf1dd669d77d"). InnerVolumeSpecName "kube-api-access-5wpwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:45:26.935535 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:26.935515    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d79cd1fc-5112-4e33-83a0-c46f038c485a-kube-api-access-69n7f" (OuterVolumeSpecName: "kube-api-access-69n7f") pod "d79cd1fc-5112-4e33-83a0-c46f038c485a" (UID: "d79cd1fc-5112-4e33-83a0-c46f038c485a"). InnerVolumeSpecName "kube-api-access-69n7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:45:27.035405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.035370    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wpwf\" (UniqueName: \"kubernetes.io/projected/57260c99-a3a6-4389-a4f7-bf1dd669d77d-kube-api-access-5wpwf\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:45:27.035405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.035396    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57260c99-a3a6-4389-a4f7-bf1dd669d77d-plugins-config-volume\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:45:27.035405 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.035405    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69n7f\" (UniqueName: \"kubernetes.io/projected/d79cd1fc-5112-4e33-83a0-c46f038c485a-kube-api-access-69n7f\") on node \"ip-10-0-141-193.ec2.internal\" DevicePath \"\""
Sep 10 15:45:27.105701 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.105665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:27.297907 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.297870    2562 generic.go:356] "Generic (PLEG): container finished" podID="57260c99-a3a6-4389-a4f7-bf1dd669d77d" containerID="655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9" exitCode=0
Sep 10 15:45:27.298086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.297916    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"
Sep 10 15:45:27.298086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.297951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j" event={"ID":"57260c99-a3a6-4389-a4f7-bf1dd669d77d","Type":"ContainerDied","Data":"655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9"}
Sep 10 15:45:27.298086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.298008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j" event={"ID":"57260c99-a3a6-4389-a4f7-bf1dd669d77d","Type":"ContainerDied","Data":"c8c0cfdf0f9243486311846c9ef0efacb1c04528afd349948e0a912ff46c9359"}
Sep 10 15:45:27.298086 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.298039    2562 scope.go:117] "RemoveContainer" containerID="655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9"
Sep 10 15:45:27.299637 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.299607    2562 generic.go:356] "Generic (PLEG): container finished" podID="d79cd1fc-5112-4e33-83a0-c46f038c485a" containerID="5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800" exitCode=0
Sep 10 15:45:27.299739 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.299650    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"
Sep 10 15:45:27.299739 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.299649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh" event={"ID":"d79cd1fc-5112-4e33-83a0-c46f038c485a","Type":"ContainerDied","Data":"5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800"}
Sep 10 15:45:27.299854 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.299772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh" event={"ID":"d79cd1fc-5112-4e33-83a0-c46f038c485a","Type":"ContainerDied","Data":"e78c79e48e25a45068d2a3cb18e0bb6273f88a511f2eba3cb89d5a47c4e25273"}
Sep 10 15:45:27.307941 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.307921    2562 scope.go:117] "RemoveContainer" containerID="655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9"
Sep 10 15:45:27.308234 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:45:27.308216    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9\": container with ID starting with 655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9 not found: ID does not exist" containerID="655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9"
Sep 10 15:45:27.308296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.308243    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9"} err="failed to get container status \"655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9\": rpc error: code = NotFound desc = could not find container \"655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9\": container with ID starting with 655a418af6c8794a116e7c0d3a3a42f0267ad08cd02d32af504afc28f0a910a9 not found: ID does not exist"
Sep 10 15:45:27.308296 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.308267    2562 scope.go:117] "RemoveContainer" containerID="5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800"
Sep 10 15:45:27.316447 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.316429    2562 scope.go:117] "RemoveContainer" containerID="5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800"
Sep 10 15:45:27.316733 ip-10-0-141-193 kubenswrapper[2562]: E0910 15:45:27.316715    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800\": container with ID starting with 5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800 not found: ID does not exist" containerID="5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800"
Sep 10 15:45:27.316784 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.316741    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800"} err="failed to get container status \"5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800\": rpc error: code = NotFound desc = could not find container \"5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800\": container with ID starting with 5274f3d0e7639f41f948d0237052b1d5e7983e6667eadae4431d16e90f370800 not found: ID does not exist"
Sep 10 15:45:27.322100 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.322075    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"]
Sep 10 15:45:27.323903 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.323877    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc3e54-74fbf459d-qgj4j"]
Sep 10 15:45:27.332317 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.332262    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"]
Sep 10 15:45:27.334414 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.334392    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc3e54-5ff8b9688f-fjpqh"]
Sep 10 15:45:27.955123 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.955041    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57260c99-a3a6-4389-a4f7-bf1dd669d77d" path="/var/lib/kubelet/pods/57260c99-a3a6-4389-a4f7-bf1dd669d77d/volumes"
Sep 10 15:45:27.955497 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.955338    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ae033d1-1ada-4943-9660-966b62e423a1" path="/var/lib/kubelet/pods/9ae033d1-1ada-4943-9660-966b62e423a1/volumes"
Sep 10 15:45:27.955688 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:27.955674    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d79cd1fc-5112-4e33-83a0-c46f038c485a" path="/var/lib/kubelet/pods/d79cd1fc-5112-4e33-83a0-c46f038c485a/volumes"
Sep 10 15:45:29.228624 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:29.228582    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:45:32.973812 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:32.973777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:33.085664 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:33.085613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:33.243771 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:33.243687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:37.421741 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:37.421698    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:45:43.360478 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:45:43.360408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:00.036151 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:46:00.036126    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:46:00.041471 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:46:00.041423    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:46:00.208345 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:46:00.208311    2562 scope.go:117] "RemoveContainer" containerID="15aee816b9bf8e09b0aaa4c4d5f7e4ee3c74297f27882d13cdd3d980b334aeff"
Sep 10 15:46:00.226806 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:46:00.226765    2562 scope.go:117] "RemoveContainer" containerID="e29e737de0ed9bc6b94d6ea0f2226b48f51be20ea0523ad447ca0eadc7dd68ba"
Sep 10 15:46:57.735408 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:46:57.735373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:58.620088 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:46:58.620050    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:46:59.498830 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:46:59.498783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:01.004248 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:47:01.004205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:23.027010 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:47:23.026923    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:47:30.816890 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:47:30.816852    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:47:38.219522 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:47:38.219482    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:47:46.519905 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:47:46.519867    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:47:53.518752 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:47:53.518710    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:48:01.618906 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:48:01.618872    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:48:08.819277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:48:08.819240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:10.650009 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:48:10.649975    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:11.817287 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:48:11.817250    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:48:44.422621 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:48:44.422586    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:44.488022 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:48:44.487985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:38.926024 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:49:38.925988    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:49:47.223718 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:49:47.223678    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:49:55.120136 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:49:55.120090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:50:00.291219 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:00.291184    2562 scope.go:117] "RemoveContainer" containerID="798bec52074afae9501c38824d96f157c72a1bfa1d64ea1a964ef1964c9174db"
Sep 10 15:50:00.299806 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:00.299782    2562 scope.go:117] "RemoveContainer" containerID="b19c55922dafa5746be81820ea9d10e91d7aa95699aefd0c32751661b779dcff"
Sep 10 15:50:04.482880 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:04.482844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:06.829535 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:06.829500    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:50:29.917717 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:29.917639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:32.633615 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:32.633576    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:50:54.100636 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:54.100599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:55.329001 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:55.328965    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:57.319099 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:50:57.319057    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:51:00.068477 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:00.068438    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:51:00.075083 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:00.075054    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dn2vx_7f76b831-7e7f-4a52-ba28-4336032a7d9c/ovn-acl-logging/0.log"
Sep 10 15:51:00.323133 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:00.323046    2562 scope.go:117] "RemoveContainer" containerID="05903a2dd5def2477c035d3969cb5f14344e5a2f27efe7e932a59995d7eee20f"
Sep 10 15:51:05.115111 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:05.115070    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:51:17.121670 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:17.121631    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:51:25.222625 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:25.221684    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:51:32.215831 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:32.215793    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:51:40.533159 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:40.533121    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:51:43.523981 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:43.523938    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:45.827714 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:45.827679    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xvjzr"]
Sep 10 15:51:53.105080 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:53.105004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:54.038749 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:54.038709    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f847f9cd-lz6z2_c07bb165-1fe4-4340-97b4-956b0e1e6d25/copy-manifests/0.log"
Sep 10 15:51:54.046012 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:54.045986    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f847f9cd-lz6z2_c07bb165-1fe4-4340-97b4-956b0e1e6d25/manager/0.log"
Sep 10 15:51:54.821267 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:54.821237    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817585-pmh2z_f5965974-57b1-45b7-8f2b-c3a732b256cc/cleanup/0.log"
Sep 10 15:51:55.995883 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:55.995853    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-sdxdh_4cf61341-b3fd-4db1-88b2-36909e556d7b/kuadrant-console-plugin/0.log"
Sep 10 15:51:56.251845 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:56.251769    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-xvjzr_a7b3a37c-4776-4f5e-9df9-e1e62928f687/limitador/0.log"
Sep 10 15:51:57.046016 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:57.045991    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bd548dcf-zczq4_bde8662d-29b3-4cfe-aed1-bacf36903c57/kube-auth-proxy/0.log"
Sep 10 15:51:57.276216 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:57.276183    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fldkc_db0c1425-6b28-4af5-8a6f-04ef280d2ddb/payload-pre-processing/0.log"
Sep 10 15:51:57.834699 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:57.834673    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rrtl9_27b680e6-7512-42e1-a898-6a621acf3821/main/0.log"
Sep 10 15:51:58.267023 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:58.266925    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwqg5r_6cc1af06-6148-4c96-b823-2a31315ff64e/main/0.log"
Sep 10 15:51:58.374566 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:58.374535    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6hvbx_57e8df30-fda5-447c-96cb-edda7fe425f0/main/0.log"
Sep 10 15:51:58.488951 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:51:58.488919    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-nzcx4_e7c8ddcb-fccf-4d77-86e1-4fe862067934/main/0.log"
Sep 10 15:52:00.345001 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:00.344964    2562 scope.go:117] "RemoveContainer" containerID="80bceab56b1fdf292206b3364d89a0ffcefff17236e96aa953d997c35a5b0f22"
Sep 10 15:52:03.915869 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:03.915832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:04.008314 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:04.008275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:06.962862 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:06.962832    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z98zv_75f17da5-a2cb-460c-b5ad-5804f13505dc/global-pull-secret-syncer/0.log"
Sep 10 15:52:07.007714 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:07.007681    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pqh9r_62582b57-cacc-4c8f-8bb7-d2b2aaa19a20/konnectivity-agent/0.log"
Sep 10 15:52:07.129109 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:07.129081    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-193.ec2.internal_756919c48f155444a9a2e4c796c6a07b/haproxy/0.log"
Sep 10 15:52:11.614914 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:11.614833    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-sdxdh_4cf61341-b3fd-4db1-88b2-36909e556d7b/kuadrant-console-plugin/0.log"
Sep 10 15:52:11.707805 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:11.707778    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-xvjzr_a7b3a37c-4776-4f5e-9df9-e1e62928f687/limitador/0.log"
Sep 10 15:52:13.299960 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.299883    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_488bb619-59df-4365-9ff1-4736a58eb8f3/alertmanager/0.log"
Sep 10 15:52:13.321914 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.321883    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_488bb619-59df-4365-9ff1-4736a58eb8f3/config-reloader/0.log"
Sep 10 15:52:13.345908 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.345882    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_488bb619-59df-4365-9ff1-4736a58eb8f3/kube-rbac-proxy-web/0.log"
Sep 10 15:52:13.370826 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.370801    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_488bb619-59df-4365-9ff1-4736a58eb8f3/kube-rbac-proxy/0.log"
Sep 10 15:52:13.397777 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.397753    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_488bb619-59df-4365-9ff1-4736a58eb8f3/kube-rbac-proxy-metric/0.log"
Sep 10 15:52:13.421165 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.421142    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_488bb619-59df-4365-9ff1-4736a58eb8f3/prom-label-proxy/0.log"
Sep 10 15:52:13.442399 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.442353    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_488bb619-59df-4365-9ff1-4736a58eb8f3/init-config-reloader/0.log"
Sep 10 15:52:13.515378 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.515350    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-cbnsb_68d4721f-7567-4739-ac9f-5a2db94bf08e/kube-state-metrics/0.log"
Sep 10 15:52:13.537362 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.537338    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-cbnsb_68d4721f-7567-4739-ac9f-5a2db94bf08e/kube-rbac-proxy-main/0.log"
Sep 10 15:52:13.569740 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.569717    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-cbnsb_68d4721f-7567-4739-ac9f-5a2db94bf08e/kube-rbac-proxy-self/0.log"
Sep 10 15:52:13.596822 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.596796    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-559bd765d7-9fdj2_08e15dc8-5946-4658-9704-d775937e989d/metrics-server/0.log"
Sep 10 15:52:13.627613 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.627584    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-kk4z4_ee6db9ec-40e9-4aa7-9452-43936d102961/monitoring-plugin/0.log"
Sep 10 15:52:13.657932 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.657886    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwjdd_da91d9d1-45cd-479b-be14-96dace47f489/node-exporter/0.log"
Sep 10 15:52:13.679246 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.679222    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwjdd_da91d9d1-45cd-479b-be14-96dace47f489/kube-rbac-proxy/0.log"
Sep 10 15:52:13.701997 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.701972    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwjdd_da91d9d1-45cd-479b-be14-96dace47f489/init-textfile/0.log"
Sep 10 15:52:13.938973 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.938888    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-gchdd_27b8e716-2ad5-429d-b692-acdceb3d02bb/kube-rbac-proxy-main/0.log"
Sep 10 15:52:13.963082 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.963053    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-gchdd_27b8e716-2ad5-429d-b692-acdceb3d02bb/kube-rbac-proxy-self/0.log"
Sep 10 15:52:13.988622 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:13.988591    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-gchdd_27b8e716-2ad5-429d-b692-acdceb3d02bb/openshift-state-metrics/0.log"
Sep 10 15:52:14.041439 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.041404    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c3ad68-8719-4295-9bbc-b3fd15cd9f83/prometheus/0.log"
Sep 10 15:52:14.073295 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.073264    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c3ad68-8719-4295-9bbc-b3fd15cd9f83/config-reloader/0.log"
Sep 10 15:52:14.111656 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.111633    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c3ad68-8719-4295-9bbc-b3fd15cd9f83/thanos-sidecar/0.log"
Sep 10 15:52:14.146875 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.146849    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c3ad68-8719-4295-9bbc-b3fd15cd9f83/kube-rbac-proxy-web/0.log"
Sep 10 15:52:14.176630 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.176604    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c3ad68-8719-4295-9bbc-b3fd15cd9f83/kube-rbac-proxy/0.log"
Sep 10 15:52:14.201810 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.201747    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c3ad68-8719-4295-9bbc-b3fd15cd9f83/kube-rbac-proxy-thanos/0.log"
Sep 10 15:52:14.229408 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.229384    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43c3ad68-8719-4295-9bbc-b3fd15cd9f83/init-config-reloader/0.log"
Sep 10 15:52:14.331473 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.331429    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-n2fs8_fde0e148-ef46-41bc-a503-0f7770774b1f/prometheus-operator-admission-webhook/0.log"
Sep 10 15:52:14.360971 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.360946    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c7466dc5-spvlq_0b0c05d6-c031-46ff-a124-6914558773b6/telemeter-client/0.log"
Sep 10 15:52:14.385262 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.385238    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c7466dc5-spvlq_0b0c05d6-c031-46ff-a124-6914558773b6/reload/0.log"
Sep 10 15:52:14.406425 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.406399    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c7466dc5-spvlq_0b0c05d6-c031-46ff-a124-6914558773b6/kube-rbac-proxy/0.log"
Sep 10 15:52:14.442851 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.442827    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55865655fd-78bnb_5726236c-0207-45cf-b08f-bc28f20796ce/thanos-query/0.log"
Sep 10 15:52:14.464812 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.464794    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55865655fd-78bnb_5726236c-0207-45cf-b08f-bc28f20796ce/kube-rbac-proxy-web/0.log"
Sep 10 15:52:14.489549 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.489524    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55865655fd-78bnb_5726236c-0207-45cf-b08f-bc28f20796ce/kube-rbac-proxy/0.log"
Sep 10 15:52:14.512919 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.512900    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55865655fd-78bnb_5726236c-0207-45cf-b08f-bc28f20796ce/prom-label-proxy/0.log"
Sep 10 15:52:14.535360 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.535339    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55865655fd-78bnb_5726236c-0207-45cf-b08f-bc28f20796ce/kube-rbac-proxy-rules/0.log"
Sep 10 15:52:14.557807 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:14.557782    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55865655fd-78bnb_5726236c-0207-45cf-b08f-bc28f20796ce/kube-rbac-proxy-metrics/0.log"
Sep 10 15:52:16.066954 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:16.066926    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-b2znc_61b5b7d9-0c2a-4c3a-a461-d8a4414b3a02/networking-console-plugin/0.log"
Sep 10 15:52:17.093169 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093133    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"]
Sep 10 15:52:17.093659 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093639    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ae033d1-1ada-4943-9660-966b62e423a1" containerName="cleanup"
Sep 10 15:52:17.093706 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093662    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ae033d1-1ada-4943-9660-966b62e423a1" containerName="cleanup"
Sep 10 15:52:17.093706 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093677    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d79cd1fc-5112-4e33-83a0-c46f038c485a" containerName="payload-processing"
Sep 10 15:52:17.093706 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093686    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d79cd1fc-5112-4e33-83a0-c46f038c485a" containerName="payload-processing"
Sep 10 15:52:17.093706 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093694    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57260c99-a3a6-4389-a4f7-bf1dd669d77d" containerName="payload-pre-processing"
Sep 10 15:52:17.093706 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093704    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="57260c99-a3a6-4389-a4f7-bf1dd669d77d" containerName="payload-pre-processing"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093719    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ba3a57f-92e5-4de4-acc1-277d1574bffe" containerName="payload-pre-processing"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093728    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ba3a57f-92e5-4de4-acc1-277d1574bffe" containerName="payload-pre-processing"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093761    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5965974-57b1-45b7-8f2b-c3a732b256cc" containerName="cleanup"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093769    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5965974-57b1-45b7-8f2b-c3a732b256cc" containerName="cleanup"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093863    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="57260c99-a3a6-4389-a4f7-bf1dd669d77d" containerName="payload-pre-processing"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093874    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ba3a57f-92e5-4de4-acc1-277d1574bffe" containerName="payload-pre-processing"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093881    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ae033d1-1ada-4943-9660-966b62e423a1" containerName="cleanup"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093886    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d79cd1fc-5112-4e33-83a0-c46f038c485a" containerName="payload-processing"
Sep 10 15:52:17.093948 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.093895    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5965974-57b1-45b7-8f2b-c3a732b256cc" containerName="cleanup"
Sep 10 15:52:17.096939 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.096922    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.099154 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.099128    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8zzjt\"/\"openshift-service-ca.crt\""
Sep 10 15:52:17.099277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.099223    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8zzjt\"/\"default-dockercfg-kqxjj\""
Sep 10 15:52:17.100118 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.100096    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8zzjt\"/\"kube-root-ca.crt\""
Sep 10 15:52:17.105404 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.105381    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"]
Sep 10 15:52:17.166682 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.166649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-podres\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.166846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.166705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-sys\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.166846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.166729    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-proc\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.166846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.166758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz5sn\" (UniqueName: \"kubernetes.io/projected/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-kube-api-access-bz5sn\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.166846 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.166788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-lib-modules\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268038 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-lib-modules\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-podres\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-sys\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-proc\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz5sn\" (UniqueName: \"kubernetes.io/projected/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-kube-api-access-bz5sn\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268213 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268196    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-lib-modules\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-sys\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-podres\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.268437 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.268266    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-proc\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.277207 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.277179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz5sn\" (UniqueName: \"kubernetes.io/projected/d6e6922b-7b01-445b-bbf3-360cc75f2ed3-kube-api-access-bz5sn\") pod \"perf-node-gather-daemonset-pfrxj\" (UID: \"d6e6922b-7b01-445b-bbf3-360cc75f2ed3\") " pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.408138 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.408055    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:17.683762 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:17.683718    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"]
Sep 10 15:52:17.686144 ip-10-0-141-193 kubenswrapper[2562]: W0910 15:52:17.686113    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd6e6922b_7b01_445b_bbf3_360cc75f2ed3.slice/crio-52726ddc59adb32270f353e79476759fa94cc6e092a8763e36e1d3d24c355a3c WatchSource:0}: Error finding container 52726ddc59adb32270f353e79476759fa94cc6e092a8763e36e1d3d24c355a3c: Status 404 returned error can't find the container with id 52726ddc59adb32270f353e79476759fa94cc6e092a8763e36e1d3d24c355a3c
Sep 10 15:52:18.397467 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:18.397409    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj" event={"ID":"d6e6922b-7b01-445b-bbf3-360cc75f2ed3","Type":"ContainerStarted","Data":"0ebe234679cd9455410c8e2694d587000bdb6dfd67e1619f53480e1ad5c686e4"}
Sep 10 15:52:18.397966 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:18.397490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj" event={"ID":"d6e6922b-7b01-445b-bbf3-360cc75f2ed3","Type":"ContainerStarted","Data":"52726ddc59adb32270f353e79476759fa94cc6e092a8763e36e1d3d24c355a3c"}
Sep 10 15:52:18.397966 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:18.397547    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:18.417928 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:18.417877    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj" podStartSLOduration=1.4178602040000001 podStartE2EDuration="1.417860204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:52:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:52:18.417487519 +0000 UTC m=+2178.953443092" watchObservedRunningTime="2026-09-10 15:52:18.417860204 +0000 UTC m=+2178.953815765"
Sep 10 15:52:19.409021 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:19.408992    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x5bkx_4983b7b7-1ffe-45a5-8cdf-117d59ec203a/dns/0.log"
Sep 10 15:52:19.431553 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:19.431527    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x5bkx_4983b7b7-1ffe-45a5-8cdf-117d59ec203a/kube-rbac-proxy/0.log"
Sep 10 15:52:19.520635 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:19.520603    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pwcph_d5749a66-bb3e-4dbf-bb99-fd2d9f8c9b00/dns-node-resolver/0.log"
Sep 10 15:52:20.074053 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:20.074023    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74b4cd8b4d-45hnw_33dca931-cc89-4713-b518-c93ef137e711/registry/0.log"
Sep 10 15:52:20.094944 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:20.094915    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2htvg_da2cfda9-d1cb-41e0-978c-8c14f669fe90/node-ca/0.log"
Sep 10 15:52:21.471064 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:21.471033    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bd548dcf-zczq4_bde8662d-29b3-4cfe-aed1-bacf36903c57/kube-auth-proxy/0.log"
Sep 10 15:52:21.542438 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:21.542409    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fldkc_db0c1425-6b28-4af5-8a6f-04ef280d2ddb/payload-pre-processing/0.log"
Sep 10 15:52:22.516215 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:22.516180    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hc4nl_8067dd20-4892-4078-aef9-b405fd646a7b/serve-healthcheck-canary/0.log"
Sep 10 15:52:23.317731 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:23.317701    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8sqc_7182f42f-a1d6-405e-bf4d-f6ffee314b6d/kube-rbac-proxy/0.log"
Sep 10 15:52:23.339112 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:23.339082    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8sqc_7182f42f-a1d6-405e-bf4d-f6ffee314b6d/exporter/0.log"
Sep 10 15:52:23.360277 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:23.360249    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8sqc_7182f42f-a1d6-405e-bf4d-f6ffee314b6d/extractor/0.log"
Sep 10 15:52:24.412509 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:24.412484    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8zzjt/perf-node-gather-daemonset-pfrxj"
Sep 10 15:52:26.233599 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:26.233570    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f847f9cd-lz6z2_c07bb165-1fe4-4340-97b4-956b0e1e6d25/manager/0.log"
Sep 10 15:52:26.260639 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:26.260614    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76f847f9cd-lz6z2_c07bb165-1fe4-4340-97b4-956b0e1e6d25/copy-manifests/0.log"
Sep 10 15:52:27.807985 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:27.807908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-578d56c89f-r248s_9f20cc7e-4737-4ce1-86a2-cba89e5e70b0/manager/0.log"
Sep 10 15:52:28.111197 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:28.111169    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bd548dcf-zczq4_bde8662d-29b3-4cfe-aed1-bacf36903c57/kube-auth-proxy/0.log"
Sep 10 15:52:28.173081 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:28.173053    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fldkc_db0c1425-6b28-4af5-8a6f-04ef280d2ddb/payload-pre-processing/0.log"
Sep 10 15:52:30.014293 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:30.014261    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bd548dcf-zczq4_bde8662d-29b3-4cfe-aed1-bacf36903c57/kube-auth-proxy/0.log"
Sep 10 15:52:30.074655 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:30.074623    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fldkc_db0c1425-6b28-4af5-8a6f-04ef280d2ddb/payload-pre-processing/0.log"
Sep 10 15:52:33.291869 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:33.291840    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-4d89n_629b4e6c-3fb5-4b92-9b56-de129175adfe/migrator/0.log"
Sep 10 15:52:33.314842 ip-10-0-141-193 kubenswrapper[2562]: I0910 15:52:33.314813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-4d89n_629b4e6c-3fb5-4b92-9b56-de129175adfe/graceful-termination/0.log"