Sep 11 11:49:43.816254 ip-10-0-130-2 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 11:49:43.816265 ip-10-0-130-2 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 11:49:43.816274 ip-10-0-130-2 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 11:49:43.816672 ip-10-0-130-2 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 11:49:55.226324 ip-10-0-130-2 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 11:49:55.226343 ip-10-0-130-2 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8e9ad7b086e54b2db69abd348b236b37 --
Sep 11 11:51:46.190742 ip-10-0-130-2 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 11:51:46.610023 ip-10-0-130-2 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:51:46.610023 ip-10-0-130-2 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 11:51:46.610023 ip-10-0-130-2 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:51:46.610023 ip-10-0-130-2 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 11:51:46.610023 ip-10-0-130-2 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:51:46.613649 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.613572    2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 11:51:46.618746 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618728    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:46.618746 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618745    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:46.618746 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618751    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618755    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618760    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618764    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618767    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618772    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618776    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618780    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618784    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618789    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618793    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618797    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618801    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618805    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618809    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618812    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618816    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618820    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618824    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618829    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:46.618935 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618833    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618836    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618841    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618845    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618849    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618853    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618857    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618862    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618866    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618871    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618876    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618880    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618885    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618889    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618893    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618898    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618904    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618911    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618917    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618921    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:46.619782 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618926    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618930    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618934    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618938    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618942    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618946    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618950    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618955    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618959    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618963    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618967    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618971    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618978    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618984    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618989    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618993    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.618997    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619001    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619005    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619009    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:46.620758 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619013    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619018    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619022    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619027    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619033    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619037    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619041    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619045    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619050    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619054    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619058    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619062    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619066    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619071    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619075    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619080    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619087    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619092    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619097    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:46.621610 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619101    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619106    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619110    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619114    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619118    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619123    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619127    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619131    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619135    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619140    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619144    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619148    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619152    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619156    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619160    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619164    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619170    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.619174    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620332    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620354    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:46.622419 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620359    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620363    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620367    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620372    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620376    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620380    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620384    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620388    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620392    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620396    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620400    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620404    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620408    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620412    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620416    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620420    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620424    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620428    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620433    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:46.623359 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620438    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620442    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620446    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620450    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620454    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620458    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620462    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620466    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620471    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620475    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620501    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620506    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620510    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620515    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620518    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620526    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620533    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620538    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620542    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620546    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:46.623904 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620551    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620555    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620559    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620564    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620569    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620573    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620577    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620581    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620586    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620590    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620594    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620598    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620602    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620606    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620613    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620618    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620622    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620626    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620630    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620634    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:46.624512 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620640    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620644    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620649    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620662    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620667    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620671    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620675    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620679    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620683    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620687    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620692    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620695    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620699    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620703    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620708    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620712    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620716    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620724    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620729    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620733    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:46.625047 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620738    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620744    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620749    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620755    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620759    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620763    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620767    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620771    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620775    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620779    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620783    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620786    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620792    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620796    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620801    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620805    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620810    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.620815    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621391    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:46.625553 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621399    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621404    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621408    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621412    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621416    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621420    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621425    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621429    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621433    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621437    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621441    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621446    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621450    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621454    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621458    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621462    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621466    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621470    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621474    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621479    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:46.626037 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621503    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621507    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621511    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621515    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621519    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621524    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621528    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621531    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621535    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621541    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621545    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621549    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621553    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621557    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621562    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621566    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621570    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621574    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621578    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621582    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:46.626637 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621587    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621591    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621595    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621599    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621603    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621607    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621611    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621616    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621621    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621625    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621630    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621634    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621638    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621642    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621646    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621650    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621655    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621664    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621670    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:46.627251 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621675    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621679    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621684    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621688    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621693    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621697    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621701    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621706    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621710    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621714    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621718    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621722    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621726    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621729    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621734    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621741    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621744    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621748    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621753    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621757    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:46.627744 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621761    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621765    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621771    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621775    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621779    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621786    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621791    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621796    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621801    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621805    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621810    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621814    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621819    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621823    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621827    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621831    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621836    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621840    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.621844    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.621948    2586 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 11:51:46.628380 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622001    2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622012    2586 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622019    2586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622025    2586 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622030    2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622036    2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622043    2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622050    2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622054    2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622060    2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622065    2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622070    2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622091    2586 flags.go:64] FLAG: --cgroup-root=""
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622097    2586 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622103    2586 flags.go:64] FLAG: --client-ca-file=""
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622108    2586 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622113    2586 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622118    2586 flags.go:64] FLAG: --cluster-domain=""
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622123    2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622128    2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622133    2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622138    2586 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622150    2586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 11:51:46.628897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622155    2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622159    2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622164    2586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622169    2586 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622173    2586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622178    2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622192    2586 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622197    2586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622203    2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622208    2586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622212    2586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622218    2586 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622222    2586 flags.go:64] FLAG: --enable-server="true"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622227    2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622234    2586 flags.go:64] FLAG: --event-burst="100"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622239    2586 flags.go:64] FLAG: --event-qps="50"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622245    2586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622250    2586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622255    2586 flags.go:64] FLAG: --eviction-hard=""
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622261    2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622265    2586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622273    2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622277    2586 flags.go:64] FLAG: --eviction-soft=""
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622282    2586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622286    2586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 11:51:46.629446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622291    2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622296    2586 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622300    2586 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622305    2586 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622309    2586 flags.go:64] FLAG: --feature-gates=""
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622315    2586 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622320    2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622325    2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622329    2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622334    2586 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622339    2586 flags.go:64] FLAG: --help="false"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622344    2586 flags.go:64] FLAG: --hostname-override="ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622349    2586 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622353    2586 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622359    2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622364    2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622370    2586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622374    2586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622379    2586 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622383    2586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622388    2586 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622392    2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622397    2586 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622403    2586 flags.go:64] FLAG: --kube-reserved=""
Sep 11 11:51:46.630097 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622408    2586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622412    2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622417    2586 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622421    2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622428    2586 flags.go:64] FLAG: --lock-file=""
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622433    2586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622437    2586 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622442    2586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622450    2586 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622455    2586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622460    2586 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622464    2586 flags.go:64] FLAG: --logging-format="text"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622469    2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622474    2586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622497    2586 flags.go:64] FLAG: --manifest-url=""
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622504    2586 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622510    2586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622515    2586 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622522    2586 flags.go:64] FLAG: --max-pods="110"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622527    2586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622532    2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622536    2586 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622541    2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622547    2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622551    2586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 11:51:46.630709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622556    2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622566    2586 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622571    2586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622575    2586 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622580    2586 flags.go:64] FLAG: --pod-cidr=""
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622585    2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622593    2586 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622597    2586 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622602    2586 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622607    2586 flags.go:64] FLAG: --port="10250"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622611    2586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622615    2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00b9df0fbe4624290"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622623    2586 flags.go:64] FLAG: --qos-reserved=""
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622628    2586 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622632    2586 flags.go:64] FLAG: --register-node="true"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622637    2586 flags.go:64] FLAG: --register-with-taints=""
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622643    2586 flags.go:64] FLAG: --registry-burst="10"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622648    2586 flags.go:64] FLAG: --registry-qps="5"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622653    2586 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622657    2586 flags.go:64] FLAG: --reserved-memory=""
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622663    2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622668    2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622673    2586 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622677    2586 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622681    2586 flags.go:64] FLAG: --runonce="false"
Sep 11 11:51:46.631319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622686    2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622692    2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622697    2586 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622702    2586 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622707    2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622711    2586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622717    2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622722    2586 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622726    2586 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622731    2586 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622735    2586 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622740    2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622744    2586 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622750    2586 flags.go:64] FLAG: --system-cgroups=""
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622754    2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622763    2586 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622767    2586 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622772    2586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622778    2586 flags.go:64] FLAG: --tls-min-version=""
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622784    2586 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622788    2586 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622793    2586 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622798    2586 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622802    2586 flags.go:64] FLAG: --v="2"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622808    2586 flags.go:64] FLAG: --version="false"
Sep 11 11:51:46.632066 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622814    2586 flags.go:64] FLAG: --vmodule=""
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622820    2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.622825    2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623016    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623023    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623027    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623032    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623036    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623041    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623045    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623049    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623053    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623060    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623066    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623073    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623080    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623085    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623089    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623093    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:46.632709 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623097    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623113    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623117    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623122    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623126    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623130    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623134    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623140    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623144    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623148    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623152    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623156    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623160    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623164    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623169    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623173    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623177    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623181    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623185    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:46.633457 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623191    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623197    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623201    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623205    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623209    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623214    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623218    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623222    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623226    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623231    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623235    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623239    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623244    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623248    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623252    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623257    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623261    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623265    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623270    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623273    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:46.634022 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623280    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623284    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623289    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623293    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623297    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623302    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623306    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623310    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623315    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623319    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623323    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623327    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623331    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623335    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623339    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623343    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623347    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623351    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623355    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623359    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:46.634536 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623363    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623367    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623371    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623375    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623381    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623385    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623390    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623394    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623398    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623402    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623406    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623410    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623416    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623420    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623424    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623428    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623432    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623436    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623440    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623444    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:46.635013 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623449    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623453    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623457    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.623461    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.624129    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.634441    2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.634457    2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.634475    2586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.634496    2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634551    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634556    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634560    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634564    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634567    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634569    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:46.635515 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634572    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634575    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634578    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634580    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634583    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634586    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634588    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634591    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634594    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634596    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634600    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634603    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634605    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634608    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634611    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634613    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634616    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634619    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634621    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:46.635932 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634624    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634626    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634629    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634632    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634635    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634638    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634641    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634643    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634646    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634648    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634651    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634654    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634657    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634659    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634661    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634664    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634666    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634669    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634672    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634674    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:46.636396 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634677    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634680    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634683    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634686    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634689    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634691    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634694    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634696    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634699    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634701    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634704    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634706    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634709    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634711    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634714    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634717    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634720    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634723    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634726    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634728    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:46.636977 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634731    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634733    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634736    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634740    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634744    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634747    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634749    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634752    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634755    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634758    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634760    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634763    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634765    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634768    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634771    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634775    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634778    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634781    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634784    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:46.637468 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634786    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634790    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634792    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634795    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634798    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634800    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634803    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634806    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634811    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634814    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634816    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634819    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634821    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634824    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634826    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:46.637950 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.634831    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634941    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634947    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634950    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634953    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634956    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634958    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634961    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634964    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634966    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634969    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634971    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634974    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634979    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634983    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634986    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634989    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634992    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634995    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.634998    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:46.638314 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635000    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635003    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635006    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635008    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635011    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635013    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635016    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635019    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635021    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635024    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635027    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635030    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635032    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635035    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635037    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635040    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635042    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635045    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635048    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635050    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:46.638822 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635053    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635055    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635058    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635060    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635063    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635065    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635068    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635071    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635074    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635077    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635079    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635082    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635084    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635087    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635090    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635092    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635095    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635097    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635100    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635103    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:46.639303 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635105    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635108    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635111    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635113    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635117    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635120    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635122    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635125    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635128    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635130    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635133    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635135    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635138    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635140    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635143    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635145    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635148    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635151    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635155    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:46.639813 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635158    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635161    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635164    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635167    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635170    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635173    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635175    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635178    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635180    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635183    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635185    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635188    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635190    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635193    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635196    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635199    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635201    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635204    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635206    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635209    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:46.640304 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:46.635211    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:46.640824 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.635216    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:51:46.640824 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.635331    2586 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 11:51:46.640824 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.637680    2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 11:51:46.640824 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.638573    2586 server.go:1019] "Starting client certificate rotation"
Sep 11 11:51:46.640824 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.638679    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:51:46.640824 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.638716    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:51:46.641845 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.641832    2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:51:46.642932 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.642916    2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:51:46.659516 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.659498    2586 log.go:25] "Validated CRI v1 runtime API"
Sep 11 11:51:46.662779 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.662766    2586 log.go:25] "Validated CRI v1 image API"
Sep 11 11:51:46.663378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.663354    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:51:46.663751 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.663734    2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 11:51:46.671186 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.671162    2586 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 84bab45d-f59f-43ae-a897-479b83e21a9e:/dev/nvme0n1p4 990ef8a2-8ee5-4d70-80ed-33256f772bf9:/dev/nvme0n1p3]
Sep 11 11:51:46.671247 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.671196    2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 11:51:46.675919 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.675815    2586 manager.go:216] Machine: {Timestamp:2026-09-11 11:51:46.675338826 +0000 UTC m=+0.386010154 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100710 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2731ed7b24044167094bf470d2a6a1 SystemUUID:ec2731ed-7b24-0441-6709-4bf470d2a6a1 BootID:8e9ad7b0-86e5-4b2d-b69a-bd348b236b37 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ad:4c:2e:04:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ad:4c:2e:04:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:e4:84:42:cf:a2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 11:51:46.675919 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.675915    2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 11:51:46.676051 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.675980    2586 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 11:51:46.677832 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.677812    2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 11:51:46.678013 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.677835    2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-2.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 11:51:46.680248 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.680225    2586 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 11:51:46.680248 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.680251    2586 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 11:51:46.680375 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.680263    2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:51:46.681272 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.681260    2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:51:46.682199 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.682183    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fwvnr"
Sep 11 11:51:46.682375 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.682366    2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 11:51:46.686670 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.686657    2586 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:51:46.686787 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.686777    2586 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 11:51:46.686823 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.686805    2586 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 11:51:46.686823 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.686815    2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 11:51:46.686896 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.686828    2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 11:51:46.686896 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.686836    2586 kubelet.go:398] "Adding apiserver pod source"
Sep 11 11:51:46.686896 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.686845    2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 11:51:46.687706 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.687692    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:51:46.687764 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.687715    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:51:46.689754 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.689736    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fwvnr"
Sep 11 11:51:46.692222 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.692203    2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 11:51:46.694945 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.694932    2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 11:51:46.694993 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.694962    2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 11:51:46.696934 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696920    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696937    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696943    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696949    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696958    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696964    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696977    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696983    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 11:51:46.696989 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696989    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 11:51:46.697199 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.696995    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 11:51:46.697199 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.697004    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 11:51:46.697199 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.697009    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 11:51:46.697879 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.697868    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 11:51:46.697879 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.697879    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 11:51:46.699466 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.699449    2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:46.700983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.700967    2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:46.701794 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.701780    2586 server.go:1268] "Started kubelet"
Sep 11 11:51:46.701939 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.701886    2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 11:51:46.702026 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.701958    2586 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 11:51:46.702376 ip-10-0-130-2 systemd[1]: Started Kubernetes Kubelet.
Sep 11 11:51:46.702560 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.702528    2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 11:51:46.703585 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.703552    2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 11:51:46.705259 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.705238    2586 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-130-2.ec2.internal" not found
Sep 11 11:51:46.705609 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.705595    2586 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 11:51:46.708897 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.708880    2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 11:51:46.710371 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.710355    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 11:51:46.711059 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.711038    2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 11:51:46.711315 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.711297    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Sep 11 11:51:46.711744 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.711725    2586 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 11:51:46.711874 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.711851    2586 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 11:51:46.711956 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.711725    2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 11:51:46.712214 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.712194    2586 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 11:51:46.712214 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.712214    2586 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 11:51:46.713801 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.713710    2586 factory.go:55] Registering systemd factory
Sep 11 11:51:46.713801 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.713743    2586 factory.go:223] Registration of the systemd container factory successfully
Sep 11 11:51:46.714265 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.714068    2586 factory.go:153] Registering CRI-O factory
Sep 11 11:51:46.714265 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.714232    2586 factory.go:223] Registration of the crio container factory successfully
Sep 11 11:51:46.714381 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.714300    2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 11:51:46.714381 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.714181    2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:46.714381 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.714322    2586 factory.go:103] Registering Raw factory
Sep 11 11:51:46.714381 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.714336    2586 manager.go:1196] Started watching for new ooms in manager
Sep 11 11:51:46.715004 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.714985    2586 manager.go:349] Starting recovery of all containers
Sep 11 11:51:46.716511 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.716474    2586 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-2.ec2.internal\" not found" node="ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.720598 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.720575    2586 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-130-2.ec2.internal" not found
Sep 11 11:51:46.720744 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.720708    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 11:51:46.726731 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.726715    2586 manager.go:354] Recovery completed
Sep 11 11:51:46.730681 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.730668    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:46.733732 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.733717    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:46.733786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.733744    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:46.733786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.733754    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:46.734265 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.734252    2586 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 11:51:46.734303 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.734268    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:46.734337 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.734267    2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 11:51:46.734379 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.734349    2586 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:51:46.734938 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.734919    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:46.734998 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.734952    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:46.734998 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.734965    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:46.736440 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.736429    2586 policy_none.go:49] "None policy: Start"
Sep 11 11:51:46.736495 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.736444    2586 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 11:51:46.736495 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.736453    2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 11:51:46.738324 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.738315    2586 policy_none.go:47] "Start"
Sep 11 11:51:46.776075 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.776060    2586 manager.go:337] "Starting Device Plugin manager"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.776092    2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.776102    2586 server.go:87] "Starting device plugin registration server"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.776365    2586 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.776378    2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.776500    2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.776582    2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.776592    2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.777240    2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.777288    2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-2.ec2.internal\" not found"
Sep 11 11:51:46.797040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.778714    2586 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-130-2.ec2.internal" not found
Sep 11 11:51:46.816961 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.816941    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 11:51:46.817030 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.816983    2586 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 11:51:46.817030 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.817002    2586 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 11:51:46.817097 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.817034    2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 11:51:46.819225 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.819205    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:46.877199 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.877136    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:46.878207 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.878185    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:46.878207 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.878209    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:46.878341 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.878231    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:46.878341 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.878253    2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.887080 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.887062    2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.887080 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:46.887081    2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-2.ec2.internal\": node \"ip-10-0-130-2.ec2.internal\" not found"
Sep 11 11:51:46.917365 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.917345    2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"]
Sep 11 11:51:46.920322 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.920306    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.922396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.922380    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.936995 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.936971    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.940783 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.940768    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:46.954175 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.954148    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:51:46.954402 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:46.954389    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:51:47.014634 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.014614    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7df19e81a5ac75c5d7184766eae49c7b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"7df19e81a5ac75c5d7184766eae49c7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.014706 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.014637    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7df19e81a5ac75c5d7184766eae49c7b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"7df19e81a5ac75c5d7184766eae49c7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.014706 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.014654    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d43279f9f2fc94f3afdb8b4c02cff3d7-config\") pod \"kube-apiserver-proxy-ip-10-0-130-2.ec2.internal\" (UID: \"d43279f9f2fc94f3afdb8b4c02cff3d7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.114884 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.114864    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7df19e81a5ac75c5d7184766eae49c7b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"7df19e81a5ac75c5d7184766eae49c7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.114961 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.114886    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7df19e81a5ac75c5d7184766eae49c7b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"7df19e81a5ac75c5d7184766eae49c7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.114961 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.114911    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d43279f9f2fc94f3afdb8b4c02cff3d7-config\") pod \"kube-apiserver-proxy-ip-10-0-130-2.ec2.internal\" (UID: \"d43279f9f2fc94f3afdb8b4c02cff3d7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.115027 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.114966    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7df19e81a5ac75c5d7184766eae49c7b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"7df19e81a5ac75c5d7184766eae49c7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.115027 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.114978    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d43279f9f2fc94f3afdb8b4c02cff3d7-config\") pod \"kube-apiserver-proxy-ip-10-0-130-2.ec2.internal\" (UID: \"d43279f9f2fc94f3afdb8b4c02cff3d7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.115027 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.114981    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7df19e81a5ac75c5d7184766eae49c7b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"7df19e81a5ac75c5d7184766eae49c7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.257614 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.257547    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.257614 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.257566    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Sep 11 11:51:47.638983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.638912    2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 11:51:47.639369 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.639084    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 11:51:47.639369 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.639084    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 11:51:47.639369 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.639100    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 11:51:47.687300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.687278    2586 apiserver.go:52] "Watching apiserver"
Sep 11 11:51:47.695555 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.695519    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 11:46:46 +0000 UTC" deadline="2028-02-12 14:44:34.434787627 +0000 UTC"
Sep 11 11:51:47.695605 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.695555    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12458h52m46.739234831s"
Sep 11 11:51:47.696234 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.696220    2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 11:51:47.697312 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.697294    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-fndk9","openshift-ovn-kubernetes/ovnkube-node-92cnc","kube-system/konnectivity-agent-22ffn","kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5pdht","openshift-image-registry/node-ca-qqt5h","openshift-multus/multus-additional-cni-plugins-fxsgk","openshift-multus/multus-msxfk","openshift-multus/network-metrics-daemon-v66kj","openshift-network-operator/iptables-alerter-46948","kube-system/global-pull-secret-syncer-xq77w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr","openshift-dns/node-resolver-tzkmp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"]
Sep 11 11:51:47.699739 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.699726    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:47.699812 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.699785    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:51:47.701850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.701836    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.704013 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.703993    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:47.704518 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.704497    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 11:51:47.705952 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.705930    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 11:51:47.705952 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.705930    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 11:51:47.706115 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.706004    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 11:51:47.706115 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.706004    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 11:51:47.706115 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.706027    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c6kvj\""
Sep 11 11:51:47.706115 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.705941    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 11:51:47.706298 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.706266    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.706712 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.706699    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 11:51:47.706767 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.706739    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 11:51:47.706799 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.706705    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v4ls5\""
Sep 11 11:51:47.708452 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.708437    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.709107 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.709074    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 11:51:47.709195 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.709153    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-557wt\""
Sep 11 11:51:47.709303 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.709289    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:51:47.710569 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.710555    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 11:51:47.710682 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.710668    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.711927 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.711071    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vmf5j\""
Sep 11 11:51:47.711927 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.711234    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 11:51:47.711927 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.711838    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 11:51:47.713675 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.713654    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p4pzz\""
Sep 11 11:51:47.713783 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.713683    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.714235 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.714216    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 11:51:47.714683 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.714665    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 11:51:47.714762 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.714681    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 11:51:47.714762 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.714685    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 11:51:47.716390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.716376    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.716696 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.716677    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 11:51:47.716817 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.716801    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 11:51:47.716904 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.716881    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qnth2\""
Sep 11 11:51:47.718661 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.718633    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.718761 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.718683    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:51:47.718841 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.718767    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:51:47.718900 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.718840    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-host\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.718900 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.718873    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-run-netns\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.718997 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.718898    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cni-binary-copy\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.718997 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.718917    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.718997 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.718962    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-kubelet\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.719133 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719014    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-systemd-units\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.719133 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719042    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-cni-netd\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.719133 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719068    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-kubernetes\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.719133 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719091    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29cd87c5-0429-4163-beed-e0522123f758-tmp\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.719133 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719115    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/34426957-e64e-412e-b12f-6c27280a60e8-konnectivity-ca\") pod \"konnectivity-agent-22ffn\" (UID: \"34426957-e64e-412e-b12f-6c27280a60e8\") " pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719140    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-cni-bin\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719164    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-run-ovn-kubernetes\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719192    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719201    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysctl-d\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719231    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-sys\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719236    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719256    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-etc-kubernetes\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719295    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z69n7\" (UniqueName: \"kubernetes.io/projected/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-kube-api-access-z69n7\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:47.719331 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719320    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-modprobe-d\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719343    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8skv\" (UniqueName: \"kubernetes.io/projected/2b40db0d-a049-4c64-ad35-463fc15f3bbc-kube-api-access-r8skv\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719368    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-etc-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719391    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysctl-conf\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719412    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-cnibin\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719434    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cnibin\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719450    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719473    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-slash\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719516    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-systemd\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719540    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2mh2\" (UniqueName: \"kubernetes.io/projected/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-kube-api-access-q2mh2\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719566    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21124859-d604-4df1-b825-d6dbfb5ecace-hosts-file\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719589    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-k8s-cni-cncf-io\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719618    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719653    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zqs6\" (UniqueName: \"kubernetes.io/projected/29cd87c5-0429-4163-beed-e0522123f758-kube-api-access-4zqs6\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719678    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-var-lib-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719701    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719723    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-lib-modules\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719741    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-cni-multus\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719787    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719830    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719857    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lgppv\""
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719868    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-var-lib-kubelet\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719912    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-cni-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719941    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18a3f084-1e1f-42d4-bdad-fe521b4cd471-cni-binary-copy\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719972    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-netns\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.719994    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-conf-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720038    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-daemon-config\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720068    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-system-cni-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720093    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-log-socket\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720131    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovnkube-config\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720145    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovn-node-metrics-cert\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720161    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysconfig\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720183    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29cd87c5-0429-4163-beed-e0522123f758-etc-tuned\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.720887 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720207    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-node-log\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720248    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-cni-bin\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720264    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-env-overrides\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720278    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovnkube-script-lib\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720300    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/34426957-e64e-412e-b12f-6c27280a60e8-agent-certs\") pod \"konnectivity-agent-22ffn\" (UID: \"34426957-e64e-412e-b12f-6c27280a60e8\") " pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720321    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-hostroot\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720336    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-multus-certs\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720361    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6svvx\" (UniqueName: \"kubernetes.io/projected/18a3f084-1e1f-42d4-bdad-fe521b4cd471-kube-api-access-6svvx\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720379    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-ovn\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720399    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21124859-d604-4df1-b825-d6dbfb5ecace-tmp-dir\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720418    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-socket-dir-parent\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720432    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-system-cni-dir\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720446    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-run\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720467    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lznwg\" (UniqueName: \"kubernetes.io/projected/21124859-d604-4df1-b825-d6dbfb5ecace-kube-api-access-lznwg\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720502    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-kubelet\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720536    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-systemd\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.721551 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.720963    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.723273 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.723257    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:47.723355 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.723314    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:51:47.723717 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.723702    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 11:51:47.723956 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.723941    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 11:51:47.724053 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.723942    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 11:51:47.724197 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.724184    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-prp2g\""
Sep 11 11:51:47.725506 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.725468    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.726043 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.726027    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:51:47.728120 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.728106    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4568k\""
Sep 11 11:51:47.728197 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.728184    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 11:51:47.728257 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.728194    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 11:51:47.728257 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.728206    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 11:51:47.746760 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.746745    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9m92v"
Sep 11 11:51:47.752393 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.752375    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9m92v"
Sep 11 11:51:47.813182 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.813169    2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 11:51:47.821414 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821399    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z69n7\" (UniqueName: \"kubernetes.io/projected/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-kube-api-access-z69n7\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:47.821530 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821421    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-modprobe-d\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.821530 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821447    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8skv\" (UniqueName: \"kubernetes.io/projected/2b40db0d-a049-4c64-ad35-463fc15f3bbc-kube-api-access-r8skv\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.821530 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821463    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-etc-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.821530 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821480    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysctl-conf\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821581    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-etc-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821603    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-cnibin\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821611    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-modprobe-d\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821625    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysctl-conf\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821633    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cnibin\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821657    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-cnibin\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821662    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821677    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cnibin\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821707    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/868bf557-a754-4bc4-92bb-54bf847ab8f7-serviceca\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821732    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-slash\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.821786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821755    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-systemd\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821803    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-systemd\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821810    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-slash\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821832    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2mh2\" (UniqueName: \"kubernetes.io/projected/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-kube-api-access-q2mh2\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821847    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21124859-d604-4df1-b825-d6dbfb5ecace-hosts-file\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821861    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-k8s-cni-cncf-io\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821890    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-k8s-cni-cncf-io\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821905    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821921    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21124859-d604-4df1-b825-d6dbfb5ecace-hosts-file\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821931    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zqs6\" (UniqueName: \"kubernetes.io/projected/29cd87c5-0429-4163-beed-e0522123f758-kube-api-access-4zqs6\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821955    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-var-lib-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.821980    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822007    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-lib-modules\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.822024    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822029    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-cni-multus\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822067    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822078    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-cni-multus\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822268 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.822093    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs podName:dfb47d93-fe8c-42eb-a7f7-0897d42c72ff nodeName:}" failed. No retries permitted until 2026-09-11 11:51:48.322071843 +0000 UTC m=+2.032743180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs") pod "network-metrics-daemon-v66kj" (UID: "dfb47d93-fe8c-42eb-a7f7-0897d42c72ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822122    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-var-lib-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822125    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-lib-modules\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822139    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822158    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822175    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-iptables-alerter-script\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822202    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-host-slash\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822226    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822243    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822257    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-var-lib-kubelet\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822283    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-openvswitch\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822321    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-cni-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822333    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-var-lib-kubelet\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822376    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-cni-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822409    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18a3f084-1e1f-42d4-bdad-fe521b4cd471-cni-binary-copy\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822453    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-netns\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822467    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-netns\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.822971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822519    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-conf-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822546    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-device-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822566    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-conf-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822572    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq6pm\" (UniqueName: \"kubernetes.io/projected/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-kube-api-access-wq6pm\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822603    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-daemon-config\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822627    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-system-cni-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822641    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-log-socket\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822661    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovnkube-config\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822693    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-log-socket\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822697    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovn-node-metrics-cert\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822722    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysconfig\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822744    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29cd87c5-0429-4163-beed-e0522123f758-etc-tuned\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822754    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-system-cni-dir\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822768    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49f88faf-4689-4ca0-a18a-4cb7d895eea8-kubelet-config\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822792    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8pnm\" (UniqueName: \"kubernetes.io/projected/ce8130f7-a034-46cd-ad71-5162fd0de18b-kube-api-access-q8pnm\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822797    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysconfig\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822819    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-node-log\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.823840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822841    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-cni-bin\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822895    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-env-overrides\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822921    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovnkube-script-lib\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822925    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-node-log\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822895    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18a3f084-1e1f-42d4-bdad-fe521b4cd471-cni-binary-copy\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822946    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jklc8\" (UniqueName: \"kubernetes.io/projected/868bf557-a754-4bc4-92bb-54bf847ab8f7-kube-api-access-jklc8\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822935    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-cni-bin\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.822978    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/34426957-e64e-412e-b12f-6c27280a60e8-agent-certs\") pod \"konnectivity-agent-22ffn\" (UID: \"34426957-e64e-412e-b12f-6c27280a60e8\") " pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823003    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-hostroot\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823026    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-multus-certs\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823050    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svvx\" (UniqueName: \"kubernetes.io/projected/18a3f084-1e1f-42d4-bdad-fe521b4cd471-kube-api-access-6svvx\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823071    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-hostroot\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823074    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-etc-selinux\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823099    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-ovn\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823120    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21124859-d604-4df1-b825-d6dbfb5ecace-tmp-dir\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823152    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-socket-dir-parent\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823175    2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823183    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-daemon-config\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.824655 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823195    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-system-cni-dir\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823226    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-run-multus-certs\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823229    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-socket-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823234    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b40db0d-a049-4c64-ad35-463fc15f3bbc-system-cni-dir\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823253    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/868bf557-a754-4bc4-92bb-54bf847ab8f7-host\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823283    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-run\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823303    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-env-overrides\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823184    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovnkube-config\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823382    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-run\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823308    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lznwg\" (UniqueName: \"kubernetes.io/projected/21124859-d604-4df1-b825-d6dbfb5ecace-kube-api-access-lznwg\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823428    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-kubelet\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823449    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-multus-socket-dir-parent\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823452    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823469    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovnkube-script-lib\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823475    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21124859-d604-4df1-b825-d6dbfb5ecace-tmp-dir\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823514    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-run-ovn\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823528    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-systemd\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823513    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-kubelet\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.825300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823554    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-host\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823566    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-systemd\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823590    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49f88faf-4689-4ca0-a18a-4cb7d895eea8-dbus\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823613    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823635    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-registration-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823615    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-host\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823655    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-run-netns\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823687    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cni-binary-copy\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823716    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823692    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-run-netns\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823748    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823782    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-kubelet\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823800    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-systemd-units\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823835    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-systemd-units\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823881    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-kubelet\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823903    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-cni-netd\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823925    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-kubernetes\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.825850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823946    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-sys-fs\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823968    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29cd87c5-0429-4163-beed-e0522123f758-tmp\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.823991    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/34426957-e64e-412e-b12f-6c27280a60e8-konnectivity-ca\") pod \"konnectivity-agent-22ffn\" (UID: \"34426957-e64e-412e-b12f-6c27280a60e8\") " pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824011    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-cni-bin\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824030    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-run-ovn-kubernetes\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824050    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysctl-d\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824068    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-sys\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824089    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-etc-kubernetes\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824165    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-etc-kubernetes\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824198    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-cni-netd\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824239    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-kubernetes\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824239    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-cni-binary-copy\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824256    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b40db0d-a049-4c64-ad35-463fc15f3bbc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824289    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18a3f084-1e1f-42d4-bdad-fe521b4cd471-host-var-lib-cni-bin\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824334    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-sys\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824425    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29cd87c5-0429-4163-beed-e0522123f758-etc-sysctl-d\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824403    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-host-run-ovn-kubernetes\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.824873    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/34426957-e64e-412e-b12f-6c27280a60e8-konnectivity-ca\") pod \"konnectivity-agent-22ffn\" (UID: \"34426957-e64e-412e-b12f-6c27280a60e8\") " pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:47.826561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.826463    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-ovn-node-metrics-cert\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.827479 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.826597    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/34426957-e64e-412e-b12f-6c27280a60e8-agent-certs\") pod \"konnectivity-agent-22ffn\" (UID: \"34426957-e64e-412e-b12f-6c27280a60e8\") " pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:47.827479 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.826639    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29cd87c5-0429-4163-beed-e0522123f758-etc-tuned\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.827479 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.826712    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29cd87c5-0429-4163-beed-e0522123f758-tmp\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.831856 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.831831    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z69n7\" (UniqueName: \"kubernetes.io/projected/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-kube-api-access-z69n7\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:47.832505 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.832459    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lznwg\" (UniqueName: \"kubernetes.io/projected/21124859-d604-4df1-b825-d6dbfb5ecace-kube-api-access-lznwg\") pod \"node-resolver-tzkmp\" (UID: \"21124859-d604-4df1-b825-d6dbfb5ecace\") " pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:47.833554 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.833536    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2mh2\" (UniqueName: \"kubernetes.io/projected/cd20463d-4e13-4c7d-8f18-9bc131b4f41f-kube-api-access-q2mh2\") pod \"ovnkube-node-92cnc\" (UID: \"cd20463d-4e13-4c7d-8f18-9bc131b4f41f\") " pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:47.833693 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.833676    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zqs6\" (UniqueName: \"kubernetes.io/projected/29cd87c5-0429-4163-beed-e0522123f758-kube-api-access-4zqs6\") pod \"tuned-5pdht\" (UID: \"29cd87c5-0429-4163-beed-e0522123f758\") " pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:47.833838 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.833821    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6svvx\" (UniqueName: \"kubernetes.io/projected/18a3f084-1e1f-42d4-bdad-fe521b4cd471-kube-api-access-6svvx\") pod \"multus-msxfk\" (UID: \"18a3f084-1e1f-42d4-bdad-fe521b4cd471\") " pod="openshift-multus/multus-msxfk"
Sep 11 11:51:47.833873 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.833823    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8skv\" (UniqueName: \"kubernetes.io/projected/2b40db0d-a049-4c64-ad35-463fc15f3bbc-kube-api-access-r8skv\") pod \"multus-additional-cni-plugins-fxsgk\" (UID: \"2b40db0d-a049-4c64-ad35-463fc15f3bbc\") " pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:47.924748 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924700    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jklc8\" (UniqueName: \"kubernetes.io/projected/868bf557-a754-4bc4-92bb-54bf847ab8f7-kube-api-access-jklc8\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.924748 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924727    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-etc-selinux\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924754    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-socket-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924768    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/868bf557-a754-4bc4-92bb-54bf847ab8f7-host\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924812    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/868bf557-a754-4bc4-92bb-54bf847ab8f7-host\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924807    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924844    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49f88faf-4689-4ca0-a18a-4cb7d895eea8-dbus\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924861    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924865    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924884    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-registration-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924900    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-socket-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.924906 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924909    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924927    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49f88faf-4689-4ca0-a18a-4cb7d895eea8-dbus\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924954    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-sys-fs\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.924971    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924966    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-registration-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924996    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/868bf557-a754-4bc4-92bb-54bf847ab8f7-serviceca\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.924926    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-etc-selinux\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.925029    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret podName:49f88faf-4689-4ca0-a18a-4cb7d895eea8 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:48.425011257 +0000 UTC m=+2.135682591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret") pod "global-pull-secret-syncer-xq77w" (UID: "49f88faf-4689-4ca0-a18a-4cb7d895eea8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925064    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-sys-fs\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925086    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-iptables-alerter-script\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925120    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-host-slash\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925138    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-device-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925172    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-host-slash\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925168    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq6pm\" (UniqueName: \"kubernetes.io/projected/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-kube-api-access-wq6pm\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925211    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce8130f7-a034-46cd-ad71-5162fd0de18b-device-dir\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925211    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49f88faf-4689-4ca0-a18a-4cb7d895eea8-kubelet-config\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925241    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49f88faf-4689-4ca0-a18a-4cb7d895eea8-kubelet-config\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:47.925378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925252    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8pnm\" (UniqueName: \"kubernetes.io/projected/ce8130f7-a034-46cd-ad71-5162fd0de18b-kube-api-access-q8pnm\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:47.925945 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925388    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/868bf557-a754-4bc4-92bb-54bf847ab8f7-serviceca\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.925945 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.925529    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-iptables-alerter-script\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.937057 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.937037    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:47.937057 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.937059    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:47.937174 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.937067    2586 projected.go:196] Error preparing data for projected volume kube-api-access-m87mk for pod openshift-network-diagnostics/network-check-target-fndk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:47.937174 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:47.937117    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk podName:42905634-6c88-439a-9b27-ccf67a851214 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:48.437101075 +0000 UTC m=+2.147772393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m87mk" (UniqueName: "kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk") pod "network-check-target-fndk9" (UID: "42905634-6c88-439a-9b27-ccf67a851214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:47.939211 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.939189    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jklc8\" (UniqueName: \"kubernetes.io/projected/868bf557-a754-4bc4-92bb-54bf847ab8f7-kube-api-access-jklc8\") pod \"node-ca-qqt5h\" (UID: \"868bf557-a754-4bc4-92bb-54bf847ab8f7\") " pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:47.940739 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.940719    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq6pm\" (UniqueName: \"kubernetes.io/projected/1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e-kube-api-access-wq6pm\") pod \"iptables-alerter-46948\" (UID: \"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e\") " pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:47.943058 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:47.943042    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8pnm\" (UniqueName: \"kubernetes.io/projected/ce8130f7-a034-46cd-ad71-5162fd0de18b-kube-api-access-q8pnm\") pod \"aws-ebs-csi-driver-node-mhxnr\" (UID: \"ce8130f7-a034-46cd-ad71-5162fd0de18b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:48.024170 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.024146    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:51:48.046640 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.046619    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:51:48.054092 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.054078    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5pdht"
Sep 11 11:51:48.063281 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.063261    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzkmp"
Sep 11 11:51:48.077368 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.077341    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21124859_d604_4df1_b825_d6dbfb5ecace.slice/crio-8b863b7266401a1c8d63b4e64cb8355e9114e052d13c2ff312c553840bb92dbf WatchSource:0}: Error finding container 8b863b7266401a1c8d63b4e64cb8355e9114e052d13c2ff312c553840bb92dbf: Status 404 returned error can't find the container with id 8b863b7266401a1c8d63b4e64cb8355e9114e052d13c2ff312c553840bb92dbf
Sep 11 11:51:48.077918 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.077897    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29cd87c5_0429_4163_beed_e0522123f758.slice/crio-ab05fb6eb9b2d8354c398ea815b342fd3cec638ca38b02bf7c59b32885747f5c WatchSource:0}: Error finding container ab05fb6eb9b2d8354c398ea815b342fd3cec638ca38b02bf7c59b32885747f5c: Status 404 returned error can't find the container with id ab05fb6eb9b2d8354c398ea815b342fd3cec638ca38b02bf7c59b32885747f5c
Sep 11 11:51:48.078605 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.078581    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43279f9f2fc94f3afdb8b4c02cff3d7.slice/crio-cb27dd8efef340fa876340ddcff91bf048b8917ba523fefbc231a78049a6ea7f WatchSource:0}: Error finding container cb27dd8efef340fa876340ddcff91bf048b8917ba523fefbc231a78049a6ea7f: Status 404 returned error can't find the container with id cb27dd8efef340fa876340ddcff91bf048b8917ba523fefbc231a78049a6ea7f
Sep 11 11:51:48.079728 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.079698    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-msxfk"
Sep 11 11:51:48.081641 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.081625    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:51:48.088124 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.088106    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a3f084_1e1f_42d4_bdad_fe521b4cd471.slice/crio-5178680be4d34af067c3c462f49d8dc7510fb6fa78f1a120baf2fe031942e1f2 WatchSource:0}: Error finding container 5178680be4d34af067c3c462f49d8dc7510fb6fa78f1a120baf2fe031942e1f2: Status 404 returned error can't find the container with id 5178680be4d34af067c3c462f49d8dc7510fb6fa78f1a120baf2fe031942e1f2
Sep 11 11:51:48.092905 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.092889    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fxsgk"
Sep 11 11:51:48.097320 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.097303    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-46948"
Sep 11 11:51:48.101074 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.101051    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b40db0d_a049_4c64_ad35_463fc15f3bbc.slice/crio-75c55474ef2fdfc88625152974aac651c11fa4b5ad8c7313e82eb25fa493c88e WatchSource:0}: Error finding container 75c55474ef2fdfc88625152974aac651c11fa4b5ad8c7313e82eb25fa493c88e: Status 404 returned error can't find the container with id 75c55474ef2fdfc88625152974aac651c11fa4b5ad8c7313e82eb25fa493c88e
Sep 11 11:51:48.105266 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.105115    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2c4d70_5fd2_4dcc_8df7_b20bb2ca555e.slice/crio-f0f7724a74cd64d0073c19fe1e28c9180f81baad510ec34f30539a88794024d3 WatchSource:0}: Error finding container f0f7724a74cd64d0073c19fe1e28c9180f81baad510ec34f30539a88794024d3: Status 404 returned error can't find the container with id f0f7724a74cd64d0073c19fe1e28c9180f81baad510ec34f30539a88794024d3
Sep 11 11:51:48.110309 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.110291    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr"
Sep 11 11:51:48.115362 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.115340    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qqt5h"
Sep 11 11:51:48.118738 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.118718    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8130f7_a034_46cd_ad71_5162fd0de18b.slice/crio-bba090e14e901d673526e183b74c1dd3b1a4c2275d32887ba0b48b2014bd3d95 WatchSource:0}: Error finding container bba090e14e901d673526e183b74c1dd3b1a4c2275d32887ba0b48b2014bd3d95: Status 404 returned error can't find the container with id bba090e14e901d673526e183b74c1dd3b1a4c2275d32887ba0b48b2014bd3d95
Sep 11 11:51:48.123360 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.123343    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868bf557_a754_4bc4_92bb_54bf847ab8f7.slice/crio-4534ca6f9abc4cf48304d83ce4148943cf97033e9da9c32a830607508297d8d3 WatchSource:0}: Error finding container 4534ca6f9abc4cf48304d83ce4148943cf97033e9da9c32a830607508297d8d3: Status 404 returned error can't find the container with id 4534ca6f9abc4cf48304d83ce4148943cf97033e9da9c32a830607508297d8d3
Sep 11 11:51:48.328217 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.328159    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:48.328303 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.328291    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:48.328356 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.328346    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs podName:dfb47d93-fe8c-42eb-a7f7-0897d42c72ff nodeName:}" failed. No retries permitted until 2026-09-11 11:51:49.328326239 +0000 UTC m=+3.038997555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs") pod "network-metrics-daemon-v66kj" (UID: "dfb47d93-fe8c-42eb-a7f7-0897d42c72ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:48.392688 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.392667    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34426957_e64e_412e_b12f_6c27280a60e8.slice/crio-d3e5662842530f58c5f1b4a69bf9aa48788cfaeb8d86819fad5a52493e93a353 WatchSource:0}: Error finding container d3e5662842530f58c5f1b4a69bf9aa48788cfaeb8d86819fad5a52493e93a353: Status 404 returned error can't find the container with id d3e5662842530f58c5f1b4a69bf9aa48788cfaeb8d86819fad5a52493e93a353
Sep 11 11:51:48.428847 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.428825    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:48.428950 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.428939    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:48.429000 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.428988    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret podName:49f88faf-4689-4ca0-a18a-4cb7d895eea8 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:49.428971946 +0000 UTC m=+3.139643263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret") pod "global-pull-secret-syncer-xq77w" (UID: "49f88faf-4689-4ca0-a18a-4cb7d895eea8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:48.467098 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.467077    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df19e81a5ac75c5d7184766eae49c7b.slice/crio-e45d836411c6243cfeab3fca00e43e81cd0eeb6056b3fca9fe782b327183ffee WatchSource:0}: Error finding container e45d836411c6243cfeab3fca00e43e81cd0eeb6056b3fca9fe782b327183ffee: Status 404 returned error can't find the container with id e45d836411c6243cfeab3fca00e43e81cd0eeb6056b3fca9fe782b327183ffee
Sep 11 11:51:48.507423 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:51:48.507383    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd20463d_4e13_4c7d_8f18_9bc131b4f41f.slice/crio-bce092f6f3e771aaed7263474047bb9b76da1e4ef91110e90159bf33761481da WatchSource:0}: Error finding container bce092f6f3e771aaed7263474047bb9b76da1e4ef91110e90159bf33761481da: Status 404 returned error can't find the container with id bce092f6f3e771aaed7263474047bb9b76da1e4ef91110e90159bf33761481da
Sep 11 11:51:48.529912 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.529890    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:48.530027 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.530009    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:48.530082 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.530034    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:48.530082 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.530048    2586 projected.go:196] Error preparing data for projected volume kube-api-access-m87mk for pod openshift-network-diagnostics/network-check-target-fndk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:48.530150 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.530104    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk podName:42905634-6c88-439a-9b27-ccf67a851214 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:49.530085809 +0000 UTC m=+3.240757130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m87mk" (UniqueName: "kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk") pod "network-check-target-fndk9" (UID: "42905634-6c88-439a-9b27-ccf67a851214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:48.751703 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.751336    2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:48.755951 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.755857    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:46:47 +0000 UTC" deadline="2028-02-21 14:58:03.380848233 +0000 UTC"
Sep 11 11:51:48.755951 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.755907    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12675h6m14.624944976s"
Sep 11 11:51:48.817876 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.817538    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:48.817876 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:48.817675    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:51:48.822329 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.822307    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:48.825734 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.825683    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzkmp" event={"ID":"21124859-d604-4df1-b825-d6dbfb5ecace","Type":"ContainerStarted","Data":"8b863b7266401a1c8d63b4e64cb8355e9114e052d13c2ff312c553840bb92dbf"}
Sep 11 11:51:48.830506 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.830466    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal" event={"ID":"d43279f9f2fc94f3afdb8b4c02cff3d7","Type":"ContainerStarted","Data":"cb27dd8efef340fa876340ddcff91bf048b8917ba523fefbc231a78049a6ea7f"}
Sep 11 11:51:48.832998 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.832971    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"bce092f6f3e771aaed7263474047bb9b76da1e4ef91110e90159bf33761481da"}
Sep 11 11:51:48.834901 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.834877    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-46948" event={"ID":"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e","Type":"ContainerStarted","Data":"f0f7724a74cd64d0073c19fe1e28c9180f81baad510ec34f30539a88794024d3"}
Sep 11 11:51:48.836951 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.836926    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-msxfk" event={"ID":"18a3f084-1e1f-42d4-bdad-fe521b4cd471","Type":"ContainerStarted","Data":"5178680be4d34af067c3c462f49d8dc7510fb6fa78f1a120baf2fe031942e1f2"}
Sep 11 11:51:48.839084 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.839037    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5pdht" event={"ID":"29cd87c5-0429-4163-beed-e0522123f758","Type":"ContainerStarted","Data":"ab05fb6eb9b2d8354c398ea815b342fd3cec638ca38b02bf7c59b32885747f5c"}
Sep 11 11:51:48.841172 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.841149    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" event={"ID":"7df19e81a5ac75c5d7184766eae49c7b","Type":"ContainerStarted","Data":"e45d836411c6243cfeab3fca00e43e81cd0eeb6056b3fca9fe782b327183ffee"}
Sep 11 11:51:48.843238 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.843181    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-22ffn" event={"ID":"34426957-e64e-412e-b12f-6c27280a60e8","Type":"ContainerStarted","Data":"d3e5662842530f58c5f1b4a69bf9aa48788cfaeb8d86819fad5a52493e93a353"}
Sep 11 11:51:48.845711 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.845685    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qqt5h" event={"ID":"868bf557-a754-4bc4-92bb-54bf847ab8f7","Type":"ContainerStarted","Data":"4534ca6f9abc4cf48304d83ce4148943cf97033e9da9c32a830607508297d8d3"}
Sep 11 11:51:48.847299 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.847275    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr" event={"ID":"ce8130f7-a034-46cd-ad71-5162fd0de18b","Type":"ContainerStarted","Data":"bba090e14e901d673526e183b74c1dd3b1a4c2275d32887ba0b48b2014bd3d95"}
Sep 11 11:51:48.849875 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.849845    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerStarted","Data":"75c55474ef2fdfc88625152974aac651c11fa4b5ad8c7313e82eb25fa493c88e"}
Sep 11 11:51:48.878675 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:48.878650    2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:49.337261 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:49.336678    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:49.337261 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.336883    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:49.337261 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.336944    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs podName:dfb47d93-fe8c-42eb-a7f7-0897d42c72ff nodeName:}" failed. No retries permitted until 2026-09-11 11:51:51.336926244 +0000 UTC m=+5.047597582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs") pod "network-metrics-daemon-v66kj" (UID: "dfb47d93-fe8c-42eb-a7f7-0897d42c72ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:49.437903 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:49.437865    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:49.438076 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.438050    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:49.438133 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.438110    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret podName:49f88faf-4689-4ca0-a18a-4cb7d895eea8 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:51.438091875 +0000 UTC m=+5.148763209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret") pod "global-pull-secret-syncer-xq77w" (UID: "49f88faf-4689-4ca0-a18a-4cb7d895eea8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:49.538389 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:49.538314    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:49.538641 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.538620    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:49.538731 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.538649    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:49.538731 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.538662    2586 projected.go:196] Error preparing data for projected volume kube-api-access-m87mk for pod openshift-network-diagnostics/network-check-target-fndk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:49.538731 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.538720    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk podName:42905634-6c88-439a-9b27-ccf67a851214 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:51.538701599 +0000 UTC m=+5.249372918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m87mk" (UniqueName: "kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk") pod "network-check-target-fndk9" (UID: "42905634-6c88-439a-9b27-ccf67a851214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:49.756212 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:49.756121    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:46:47 +0000 UTC" deadline="2028-03-02 18:23:19.381056171 +0000 UTC"
Sep 11 11:51:49.756212 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:49.756162    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12918h31m29.624898739s"
Sep 11 11:51:49.817821 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:49.817793    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:49.817983 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.817923    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:51:49.818324 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:49.818302    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:49.818430 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:49.818398    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:51:50.819676 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:50.819602    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:50.820120 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:50.819740    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:51:51.351978 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:51.351371    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:51.351978 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.351549    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:51.351978 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.351609    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs podName:dfb47d93-fe8c-42eb-a7f7-0897d42c72ff nodeName:}" failed. No retries permitted until 2026-09-11 11:51:55.351591142 +0000 UTC m=+9.062262476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs") pod "network-metrics-daemon-v66kj" (UID: "dfb47d93-fe8c-42eb-a7f7-0897d42c72ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:51.452291 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:51.452248    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:51.457507 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.454527    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:51.457507 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.454611    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret podName:49f88faf-4689-4ca0-a18a-4cb7d895eea8 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:55.454590964 +0000 UTC m=+9.165262295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret") pod "global-pull-secret-syncer-xq77w" (UID: "49f88faf-4689-4ca0-a18a-4cb7d895eea8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:51.556170 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:51.555566    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:51.556170 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.555753    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:51.556170 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.555775    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:51.556170 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.555789    2586 projected.go:196] Error preparing data for projected volume kube-api-access-m87mk for pod openshift-network-diagnostics/network-check-target-fndk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:51.556170 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.555845    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk podName:42905634-6c88-439a-9b27-ccf67a851214 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:55.555827168 +0000 UTC m=+9.266498506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m87mk" (UniqueName: "kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk") pod "network-check-target-fndk9" (UID: "42905634-6c88-439a-9b27-ccf67a851214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:51.817693 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:51.817612    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:51.817846 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.817750    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:51:51.818143 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:51.818121    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:51.818326 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:51.818223    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:51:52.819687 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:52.819654    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:52.820133 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:52.819773    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:51:53.817380 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:53.817350    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:53.817597 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:53.817476    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:51:53.817810 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:53.817315    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:53.817923 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:53.817903    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:51:54.818197 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:54.818168    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:54.818671 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:54.818301    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:51:55.386625 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:55.386594    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:55.386788 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.386766    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:55.386858 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.386839    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs podName:dfb47d93-fe8c-42eb-a7f7-0897d42c72ff nodeName:}" failed. No retries permitted until 2026-09-11 11:52:03.386818481 +0000 UTC m=+17.097489799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs") pod "network-metrics-daemon-v66kj" (UID: "dfb47d93-fe8c-42eb-a7f7-0897d42c72ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:55.487148 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:55.487114    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:55.487320 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.487298    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:55.487378 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.487356    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret podName:49f88faf-4689-4ca0-a18a-4cb7d895eea8 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:03.487338807 +0000 UTC m=+17.198010148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret") pod "global-pull-secret-syncer-xq77w" (UID: "49f88faf-4689-4ca0-a18a-4cb7d895eea8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:55.587862 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:55.587833    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:55.588035 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.587986    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:55.588035 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.588004    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:55.588035 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.588016    2586 projected.go:196] Error preparing data for projected volume kube-api-access-m87mk for pod openshift-network-diagnostics/network-check-target-fndk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:55.588191 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.588072    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk podName:42905634-6c88-439a-9b27-ccf67a851214 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:03.588052836 +0000 UTC m=+17.298724169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m87mk" (UniqueName: "kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk") pod "network-check-target-fndk9" (UID: "42905634-6c88-439a-9b27-ccf67a851214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:55.818237 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:55.818097    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:55.818237 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:55.818148    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:55.819043 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.818244    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:51:55.819043 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:55.818650    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:51:56.818975 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:56.818516    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:56.818975 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:56.818627    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:51:57.817475 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:57.817436    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:57.817667 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:57.817498    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:57.817667 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:57.817612    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:51:57.817773 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:57.817738    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:51:58.817674 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:58.817643    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:51:58.818091 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:58.817751    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:51:59.817687 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:59.817620    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:51:59.817816 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:59.817724    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:51:59.817816 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:51:59.817778    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:51:59.818117 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:51:59.817858    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:00.818086 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:00.818044    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:00.818681 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:00.818164    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:01.817467 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:01.817430    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:01.817467 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:01.817460    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:01.817710 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:01.817583    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:01.817758 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:01.817712    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:02.817585 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:02.817554    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:02.818020 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:02.817674    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:03.444988 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:03.444966    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:03.445113 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.445097    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:52:03.445166 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.445157    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs podName:dfb47d93-fe8c-42eb-a7f7-0897d42c72ff nodeName:}" failed. No retries permitted until 2026-09-11 11:52:19.445141939 +0000 UTC m=+33.155813256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs") pod "network-metrics-daemon-v66kj" (UID: "dfb47d93-fe8c-42eb-a7f7-0897d42c72ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:52:03.545715 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:03.545686    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:03.545841 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.545795    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:52:03.545884 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.545848    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret podName:49f88faf-4689-4ca0-a18a-4cb7d895eea8 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:19.545832343 +0000 UTC m=+33.256503659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret") pod "global-pull-secret-syncer-xq77w" (UID: "49f88faf-4689-4ca0-a18a-4cb7d895eea8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:52:03.646412 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:03.646379    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:03.646576 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.646561    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:52:03.646617 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.646580    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:52:03.646617 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.646590    2586 projected.go:196] Error preparing data for projected volume kube-api-access-m87mk for pod openshift-network-diagnostics/network-check-target-fndk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:52:03.646697 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.646641    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk podName:42905634-6c88-439a-9b27-ccf67a851214 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:19.646624192 +0000 UTC m=+33.357295531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m87mk" (UniqueName: "kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk") pod "network-check-target-fndk9" (UID: "42905634-6c88-439a-9b27-ccf67a851214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:52:03.817580 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:03.817508    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:03.817809 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.817634    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:03.817809 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:03.817697    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:03.817809 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:03.817802    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:04.818513 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:04.818261    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:04.818513 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:04.818380    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:05.817933 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:05.817897    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:05.818123 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:05.818038    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:05.818123 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:05.818092    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:05.818242 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:05.818224    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:06.818379 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:06.818340    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:06.818829 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:06.818445    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:07.818182 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.817962    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:07.818318 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:07.818293    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:07.818788 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.818767    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:07.819335 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:07.818885    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:07.907416 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.907372    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5pdht" event={"ID":"29cd87c5-0429-4163-beed-e0522123f758","Type":"ContainerStarted","Data":"d4eb9fbcb685784dc00dc5648201b0e92d56e93cacac5104ae59899750a2d76c"}
Sep 11 11:52:07.910975 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.910596    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal" event={"ID":"d43279f9f2fc94f3afdb8b4c02cff3d7","Type":"ContainerStarted","Data":"48d9179c86ced18e4efa469db138975a5b0e79ec83b4b70dbdb6fecfedc0c3ea"}
Sep 11 11:52:07.916583 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.916556    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 11:52:07.918084 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.918053    2586 generic.go:356] "Generic (PLEG): container finished" podID="cd20463d-4e13-4c7d-8f18-9bc131b4f41f" containerID="35be778205baac11587a2cd3483572afccee8c79f8c9ee3febc6bcea4b0f8b93" exitCode=1
Sep 11 11:52:07.918176 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.918141    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"37ae1c47bb94936e2810104fb1518dedcdb2e34feb812ffc08683a0cc619556e"}
Sep 11 11:52:07.918759 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.918213    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerDied","Data":"35be778205baac11587a2cd3483572afccee8c79f8c9ee3febc6bcea4b0f8b93"}
Sep 11 11:52:07.918759 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.918265    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"f9b662ec40f548343ed8f02cc5e85fafeda7cb65e38a7163db9d4805e8d0e950"}
Sep 11 11:52:07.921103 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.921077    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-msxfk" event={"ID":"18a3f084-1e1f-42d4-bdad-fe521b4cd471","Type":"ContainerStarted","Data":"9648b176636e0ec4605203ffb2167e8efe8285a5cc275e9ee675fb6e787d5ae0"}
Sep 11 11:52:07.924193 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.924147    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5pdht" podStartSLOduration=2.626083639 podStartE2EDuration="21.924133215s" totalImagesPullingTime="19.298049576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:07.923977185 +0000 UTC m=+21.634648535" watchObservedRunningTime="2026-09-11 11:52:07.924133215 +0000 UTC m=+21.634804552"
Sep 11 11:52:07.940801 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:07.940755    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-msxfk" podStartSLOduration=2.271619363 podStartE2EDuration="21.94073492s" totalImagesPullingTime="19.669115557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:07.939427885 +0000 UTC m=+21.650099238" watchObservedRunningTime="2026-09-11 11:52:07.94073492 +0000 UTC m=+21.651406258"
Sep 11 11:52:08.817645 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.817577    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:08.817791 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:08.817703    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:08.927585 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.926629    2586 generic.go:356] "Generic (PLEG): container finished" podID="7df19e81a5ac75c5d7184766eae49c7b" containerID="25f9836286a18fa09bdeed1c30c68962f2ae6b949d6ed02c3a021ca674628f48" exitCode=0
Sep 11 11:52:08.927585 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.926742    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" event={"ID":"7df19e81a5ac75c5d7184766eae49c7b","Type":"ContainerDied","Data":"25f9836286a18fa09bdeed1c30c68962f2ae6b949d6ed02c3a021ca674628f48"}
Sep 11 11:52:08.929858 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.929828    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-22ffn" event={"ID":"34426957-e64e-412e-b12f-6c27280a60e8","Type":"ContainerStarted","Data":"ba24e7fd94cb1f31bd88d754cbefd3c490ff494b7923e04e3ffa07fea1723473"}
Sep 11 11:52:08.931449 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.931421    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qqt5h" event={"ID":"868bf557-a754-4bc4-92bb-54bf847ab8f7","Type":"ContainerStarted","Data":"7c3fff83a14407279c9e4d6288f837cec5855d4efd18c55796872d9f9f27278d"}
Sep 11 11:52:08.932967 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.932944    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr" event={"ID":"ce8130f7-a034-46cd-ad71-5162fd0de18b","Type":"ContainerStarted","Data":"5d9a6131067542a893890883a50816b403db0312740d87452cdf8ce898d53012"}
Sep 11 11:52:08.934687 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.934653    2586 generic.go:356] "Generic (PLEG): container finished" podID="2b40db0d-a049-4c64-ad35-463fc15f3bbc" containerID="94b1dc4a15c7cd40900a1e040bf1e0cc2d30989ad7db047cb9a08910604996a0" exitCode=0
Sep 11 11:52:08.934796 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.934680    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerDied","Data":"94b1dc4a15c7cd40900a1e040bf1e0cc2d30989ad7db047cb9a08910604996a0"}
Sep 11 11:52:08.936422 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.936381    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzkmp" event={"ID":"21124859-d604-4df1-b825-d6dbfb5ecace","Type":"ContainerStarted","Data":"9e21632162178f062766225fc83a66dd6767f39070254a1e2dfbcc22dce7fbce"}
Sep 11 11:52:08.939587 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.939568    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 11:52:08.940149 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.940124    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"869d04c33c0693002b24eb95e7fbdb7b7aae849c52e414f95f4bcdbc3e37a183"}
Sep 11 11:52:08.940252 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.940195    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"3da4212ff66e3b6c8d232b88b9b8c92d7d36960269f24021421fb2c6e9a53f5b"}
Sep 11 11:52:08.940252 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.940230    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"43ca805cc0c1687ae53a1725b2098cb84d95ee632b1b5b4fe5a0727baf42ebf7"}
Sep 11 11:52:08.943452 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.943419    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal" podStartSLOduration=22.943407371 podStartE2EDuration="22.943407371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:07.955591343 +0000 UTC m=+21.666262686" watchObservedRunningTime="2026-09-11 11:52:08.943407371 +0000 UTC m=+22.654078700"
Sep 11 11:52:08.977419 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.977379    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-22ffn" podStartSLOduration=3.990848338 podStartE2EDuration="22.97736679s" totalImagesPullingTime="18.986518452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:08.977058182 +0000 UTC m=+22.687729530" watchObservedRunningTime="2026-09-11 11:52:08.97736679 +0000 UTC m=+22.688038128"
Sep 11 11:52:08.991329 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:08.991290    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tzkmp" podStartSLOduration=3.694836423 podStartE2EDuration="22.991273787s" totalImagesPullingTime="19.296437364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:08.990906403 +0000 UTC m=+22.701577742" watchObservedRunningTime="2026-09-11 11:52:08.991273787 +0000 UTC m=+22.701945128"
Sep 11 11:52:09.110820 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.110787    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:09.165102 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.165077    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:09.372513 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.372474    2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 11:52:09.786091 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.785940    2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T11:52:09.372509323Z","UUID":"b4710c7c-7836-48de-bde9-cbffa5592cb7","Handler":null,"Name":"","Endpoint":""}
Sep 11 11:52:09.788927 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.788904    2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 11:52:09.789053 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.788941    2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 11:52:09.817811 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.817783    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:09.817948 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:09.817902    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:09.818023 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.817954    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:09.818077 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:09.818059    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:09.943949 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.943917    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" event={"ID":"7df19e81a5ac75c5d7184766eae49c7b","Type":"ContainerStarted","Data":"75be3e44a36b58d3818a3901ab0e6777f6fba8cf5ea6006b3bdfcb532386969c"}
Sep 11 11:52:09.945728 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.945704    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr" event={"ID":"ce8130f7-a034-46cd-ad71-5162fd0de18b","Type":"ContainerStarted","Data":"d21c5ec1514766ccff95e8aa2525d5ad2f48ee22f35de706a1ee189367240f03"}
Sep 11 11:52:09.948993 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.948971    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-46948" event={"ID":"1a2c4d70-5fd2-4dcc-8df7-b20bb2ca555e","Type":"ContainerStarted","Data":"57382274e1bc928b5c021ed6ec1e911a6b2098e9d179780864ca437c0fa9856c"}
Sep 11 11:52:09.958798 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.958763    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qqt5h" podStartSLOduration=3.816043489 podStartE2EDuration="22.958749739s" totalImagesPullingTime="19.14270625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:09.036234597 +0000 UTC m=+22.746905936" watchObservedRunningTime="2026-09-11 11:52:09.958749739 +0000 UTC m=+23.669421128"
Sep 11 11:52:09.959211 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.959184    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" podStartSLOduration=23.959174271 podStartE2EDuration="23.959174271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:09.958877412 +0000 UTC m=+23.669548744" watchObservedRunningTime="2026-09-11 11:52:09.959174271 +0000 UTC m=+23.669845608"
Sep 11 11:52:09.973077 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:09.973041    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-46948" podStartSLOduration=3.790696291 podStartE2EDuration="22.973029824s" totalImagesPullingTime="19.182333533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:09.972744502 +0000 UTC m=+23.683415841" watchObservedRunningTime="2026-09-11 11:52:09.973029824 +0000 UTC m=+23.683701169"
Sep 11 11:52:10.091017 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.090896    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:10.190843 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.190807    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:10.820689 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.820521    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:10.820825 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:10.820755    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:10.830566 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.830542    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:10.879527 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.879445    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:10.953381 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.953350    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr" event={"ID":"ce8130f7-a034-46cd-ad71-5162fd0de18b","Type":"ContainerStarted","Data":"3e72a31b85d1b4867fb42f149d34909d4721505044f3e64de305f2e1ed5c4405"}
Sep 11 11:52:10.957140 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.957122    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 11:52:10.957564 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.957544    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"b48b20830e94065c459400105c11419892cfb7b3c2f320235d577e63dbda44ca"}
Sep 11 11:52:10.971553 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:10.971512    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mhxnr" podStartSLOduration=2.360903687 podStartE2EDuration="23.971474813s" totalImagesPullingTime="21.610571126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:10.97103235 +0000 UTC m=+24.681703689" watchObservedRunningTime="2026-09-11 11:52:10.971474813 +0000 UTC m=+24.682146151"
Sep 11 11:52:11.409496 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:11.409442    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:11.461835 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:11.461793    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:11.817831 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:11.817759    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:11.818001 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:11.817759    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:11.818001 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:11.817861    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:11.818001 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:11.817956    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:12.795913 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.795828    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:52:12.796557 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.796497    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:52:12.818162 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.817971    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:12.818162 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:12.818092    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:12.966079 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.966052    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 11:52:12.966560 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.966521    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"61e64618fa34b4250a2101653777c8483cf44cdb46c77dff5a360f2e6a8a5083"}
Sep 11 11:52:12.966897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.966814    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:52:12.967009 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.966942    2586 scope.go:117] "RemoveContainer" containerID="35be778205baac11587a2cd3483572afccee8c79f8c9ee3febc6bcea4b0f8b93"
Sep 11 11:52:12.983737 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:12.983713    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:52:13.315403 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:13.315164    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:52:13.315864 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:13.315839    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-22ffn"
Sep 11 11:52:13.372974 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:13.372944    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:13.817993 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:13.817947    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:13.817993 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:13.817976    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:13.818513 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:13.818099    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:13.818513 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:13.818370    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:14.736843 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.736817    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v66kj"]
Sep 11 11:52:14.736976 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.736917    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:14.737034 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:14.737015    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:14.737188 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.737164    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xq77w"]
Sep 11 11:52:14.737260 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.737238    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:14.737337 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:14.737320    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:14.748020 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.747989    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fndk9"]
Sep 11 11:52:14.748143 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.748065    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:14.748143 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:14.748125    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:14.972848 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.972752    2586 generic.go:356] "Generic (PLEG): container finished" podID="2b40db0d-a049-4c64-ad35-463fc15f3bbc" containerID="cb0b5316ca4901efa95edca0c4a75ac2a74a60ea2fe058d98a5911c2ad63d8ed" exitCode=0
Sep 11 11:52:14.973616 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.972840    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerDied","Data":"cb0b5316ca4901efa95edca0c4a75ac2a74a60ea2fe058d98a5911c2ad63d8ed"}
Sep 11 11:52:14.978754 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.978737    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 11:52:14.979103 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.979084    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" event={"ID":"cd20463d-4e13-4c7d-8f18-9bc131b4f41f","Type":"ContainerStarted","Data":"ba38f7a56732373f17a4ee0b6c5ad49adbbe8c95315b3e168a3793fff56dd2c2"}
Sep 11 11:52:14.979197 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.979143    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 11:52:14.979348 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.979333    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:52:14.994907 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:14.994890    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:52:15.024222 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:15.024180    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" podStartSLOduration=10.122375534 podStartE2EDuration="29.024167381s" totalImagesPullingTime="18.901791847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:15.022630562 +0000 UTC m=+28.733301900" watchObservedRunningTime="2026-09-11 11:52:15.024167381 +0000 UTC m=+28.734838712"
Sep 11 11:52:15.817196 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:15.817167    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:15.817196 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:15.817189    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:15.817419 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:15.817272    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:15.817419 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:15.817355    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:15.981634 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:15.981600    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 11:52:16.818708 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:16.818408    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:16.818833 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:16.818750    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:16.987381 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:16.987352    2586 generic.go:356] "Generic (PLEG): container finished" podID="2b40db0d-a049-4c64-ad35-463fc15f3bbc" containerID="5348d1ddddb8ec8a5eed512449f945c989dacb4f0c3a66b58cb92e2756ba45fd" exitCode=0
Sep 11 11:52:16.987842 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:16.987428    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerDied","Data":"5348d1ddddb8ec8a5eed512449f945c989dacb4f0c3a66b58cb92e2756ba45fd"}
Sep 11 11:52:16.987842 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:16.987604    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 11:52:17.565994 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:17.565917    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:52:17.576212 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:17.576176    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" podUID="cd20463d-4e13-4c7d-8f18-9bc131b4f41f" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 11 11:52:17.584576 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:17.584541    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" podUID="cd20463d-4e13-4c7d-8f18-9bc131b4f41f" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 11 11:52:17.818191 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:17.818132    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:17.818297 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:17.818132    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:17.818297 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:17.818229    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:17.818366 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:17.818292    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:17.992422 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:17.992388    2586 generic.go:356] "Generic (PLEG): container finished" podID="2b40db0d-a049-4c64-ad35-463fc15f3bbc" containerID="0198bfdbc0186c5bd88260681cbe028c38febbd8c95079ef69eb8dfc4ee63a95" exitCode=0
Sep 11 11:52:17.992892 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:17.992454    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerDied","Data":"0198bfdbc0186c5bd88260681cbe028c38febbd8c95079ef69eb8dfc4ee63a95"}
Sep 11 11:52:18.003142 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:18.003110    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc" podUID="cd20463d-4e13-4c7d-8f18-9bc131b4f41f" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 11 11:52:18.818158 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:18.818071    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:18.818307 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:18.818236    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq77w" podUID="49f88faf-4689-4ca0-a18a-4cb7d895eea8"
Sep 11 11:52:19.467971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:19.467941    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:19.468414 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.468050    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:52:19.468414 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.468116    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs podName:dfb47d93-fe8c-42eb-a7f7-0897d42c72ff nodeName:}" failed. No retries permitted until 2026-09-11 11:52:51.468095969 +0000 UTC m=+65.178767292 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs") pod "network-metrics-daemon-v66kj" (UID: "dfb47d93-fe8c-42eb-a7f7-0897d42c72ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:52:19.568580 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:19.568536    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:19.568744 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.568727    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:52:19.568810 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.568799    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret podName:49f88faf-4689-4ca0-a18a-4cb7d895eea8 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:51.568780779 +0000 UTC m=+65.279452110 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret") pod "global-pull-secret-syncer-xq77w" (UID: "49f88faf-4689-4ca0-a18a-4cb7d895eea8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:52:19.668905 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:19.668876    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:19.669083 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.669012    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:52:19.669083 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.669035    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:52:19.669083 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.669049    2586 projected.go:196] Error preparing data for projected volume kube-api-access-m87mk for pod openshift-network-diagnostics/network-check-target-fndk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:52:19.669193 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.669108    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk podName:42905634-6c88-439a-9b27-ccf67a851214 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:51.669089637 +0000 UTC m=+65.379760956 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m87mk" (UniqueName: "kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk") pod "network-check-target-fndk9" (UID: "42905634-6c88-439a-9b27-ccf67a851214") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:52:19.818356 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:19.818178    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:19.818356 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.818328    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fndk9" podUID="42905634-6c88-439a-9b27-ccf67a851214"
Sep 11 11:52:19.818589 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:19.818361    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:19.818589 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:19.818464    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v66kj" podUID="dfb47d93-fe8c-42eb-a7f7-0897d42c72ff"
Sep 11 11:52:20.575375 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.575344    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeReady"
Sep 11 11:52:20.575785 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.575477    2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 11:52:20.617627 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.617599    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ss9wb"]
Sep 11 11:52:20.630775 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.630751    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l68qh"]
Sep 11 11:52:20.630948 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.630903    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.633877 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.633857    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 11:52:20.633990 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.633893    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 11:52:20.633990 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.633936    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mvnq6\""
Sep 11 11:52:20.641718 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.641701    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ss9wb"]
Sep 11 11:52:20.641793 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.641731    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l68qh"]
Sep 11 11:52:20.641829 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.641804    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:20.644280 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.644264    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 11:52:20.644664 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.644645    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 11:52:20.644749 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.644655    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5ncdh\""
Sep 11 11:52:20.644749 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.644710    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 11:52:20.715310 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.715263    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:20.777540 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.777505    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71cca530-f27e-4d2a-8a03-e6a7d99064a5-cert\") pod \"ingress-canary-l68qh\" (UID: \"71cca530-f27e-4d2a-8a03-e6a7d99064a5\") " pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:20.777740 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.777552    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsngr\" (UniqueName: \"kubernetes.io/projected/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-kube-api-access-nsngr\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.777740 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.777602    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-config-volume\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.777740 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.777626    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxtgx\" (UniqueName: \"kubernetes.io/projected/71cca530-f27e-4d2a-8a03-e6a7d99064a5-kube-api-access-dxtgx\") pod \"ingress-canary-l68qh\" (UID: \"71cca530-f27e-4d2a-8a03-e6a7d99064a5\") " pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:20.777740 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.777684    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.777740 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.777709    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-tmp-dir\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.817915 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.817843    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:20.820721 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.820687    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:52:20.878219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878192    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-config-volume\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.878219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878223    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxtgx\" (UniqueName: \"kubernetes.io/projected/71cca530-f27e-4d2a-8a03-e6a7d99064a5-kube-api-access-dxtgx\") pod \"ingress-canary-l68qh\" (UID: \"71cca530-f27e-4d2a-8a03-e6a7d99064a5\") " pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:20.878394 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878276    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.878394 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878299    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-tmp-dir\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.878466 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:20.878424    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:20.878533 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:20.878496    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls podName:2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:21.37846232 +0000 UTC m=+35.089133654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls") pod "dns-default-ss9wb" (UID: "2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:20.878533 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878518    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71cca530-f27e-4d2a-8a03-e6a7d99064a5-cert\") pod \"ingress-canary-l68qh\" (UID: \"71cca530-f27e-4d2a-8a03-e6a7d99064a5\") " pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:20.878643 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878553    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsngr\" (UniqueName: \"kubernetes.io/projected/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-kube-api-access-nsngr\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.878696 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878649    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-tmp-dir\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.878816 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.878799    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-config-volume\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.883096 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.883068    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71cca530-f27e-4d2a-8a03-e6a7d99064a5-cert\") pod \"ingress-canary-l68qh\" (UID: \"71cca530-f27e-4d2a-8a03-e6a7d99064a5\") " pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:20.887524 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.887503    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsngr\" (UniqueName: \"kubernetes.io/projected/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-kube-api-access-nsngr\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:20.887742 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.887726    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxtgx\" (UniqueName: \"kubernetes.io/projected/71cca530-f27e-4d2a-8a03-e6a7d99064a5-kube-api-access-dxtgx\") pod \"ingress-canary-l68qh\" (UID: \"71cca530-f27e-4d2a-8a03-e6a7d99064a5\") " pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:20.949772 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:20.949743    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l68qh"
Sep 11 11:52:21.294943 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.294850    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:21.381832 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.381800    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:21.382022 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:21.381965    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:21.382090 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:21.382045    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls podName:2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:22.382019469 +0000 UTC m=+36.092690799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls") pod "dns-default-ss9wb" (UID: "2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:21.818143 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.818091    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:21.818537 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.818103    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:21.822793 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.822737    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:52:21.822922 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.822411    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 11:52:21.822922 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.822821    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 11:52:21.825659 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.823147    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f8bwk\""
Sep 11 11:52:21.825659 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:21.823179    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-74fqm\""
Sep 11 11:52:22.388853 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:22.388696    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:22.388999 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:22.388852    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:22.388999 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:22.388918    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls podName:2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:24.388900574 +0000 UTC m=+38.099571911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls") pod "dns-default-ss9wb" (UID: "2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:22.513733 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:22.513665    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l68qh"]
Sep 11 11:52:22.518452 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:22.518421    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cca530_f27e_4d2a_8a03_e6a7d99064a5.slice/crio-02d6a067b8b571447ac70aa5e0cc618fbbf4c10894c90faa726f15e30a0609eb WatchSource:0}: Error finding container 02d6a067b8b571447ac70aa5e0cc618fbbf4c10894c90faa726f15e30a0609eb: Status 404 returned error can't find the container with id 02d6a067b8b571447ac70aa5e0cc618fbbf4c10894c90faa726f15e30a0609eb
Sep 11 11:52:23.004476 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:23.004446    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l68qh" event={"ID":"71cca530-f27e-4d2a-8a03-e6a7d99064a5","Type":"ContainerStarted","Data":"02d6a067b8b571447ac70aa5e0cc618fbbf4c10894c90faa726f15e30a0609eb"}
Sep 11 11:52:23.007241 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:23.007219    2586 generic.go:356] "Generic (PLEG): container finished" podID="2b40db0d-a049-4c64-ad35-463fc15f3bbc" containerID="833e04bf1ec75a58a008634f6ba2323e0e59033a924b6ced5edba7f91b1fc490" exitCode=0
Sep 11 11:52:23.007300 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:23.007253    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerDied","Data":"833e04bf1ec75a58a008634f6ba2323e0e59033a924b6ced5edba7f91b1fc490"}
Sep 11 11:52:24.013413 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:24.013373    2586 generic.go:356] "Generic (PLEG): container finished" podID="2b40db0d-a049-4c64-ad35-463fc15f3bbc" containerID="7bff5ab8922ea11f2ed303b2e24b841e2854800fe0a4a61649fa11a0ff82dcd9" exitCode=0
Sep 11 11:52:24.013802 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:24.013439    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerDied","Data":"7bff5ab8922ea11f2ed303b2e24b841e2854800fe0a4a61649fa11a0ff82dcd9"}
Sep 11 11:52:24.406799 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:24.406718    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:24.406947 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:24.406878    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:24.407013 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:24.406956    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls podName:2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:28.406936919 +0000 UTC m=+42.117608240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls") pod "dns-default-ss9wb" (UID: "2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:25.019992 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.019924    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" event={"ID":"2b40db0d-a049-4c64-ad35-463fc15f3bbc","Type":"ContainerStarted","Data":"59d57a0b76be412fd7617046c9ade3b3fc959badb74623de6da48f4d00278e88"}
Sep 11 11:52:25.047145 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.047093    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fxsgk" podStartSLOduration=6.732308453 podStartE2EDuration="38.047074159s" totalImagesPullingTime="31.314765706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:25.046927565 +0000 UTC m=+38.757598903" watchObservedRunningTime="2026-09-11 11:52:25.047074159 +0000 UTC m=+38.757745475"
Sep 11 11:52:25.136125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.136097    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:25.239375 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.239180    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:25.336700 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.336580    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:25.437144 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.437108    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:25.907732 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.907699    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:25.971329 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:25.971300    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:26.023245 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:26.023212    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l68qh" event={"ID":"71cca530-f27e-4d2a-8a03-e6a7d99064a5","Type":"ContainerStarted","Data":"06879c9f34f8a2f7634a9a8f42417dd5af4e10ef08beafcd4f29c002b21fc0cc"}
Sep 11 11:52:26.040558 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:26.040518    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l68qh" podStartSLOduration=3.442611281 podStartE2EDuration="6.040503858s" totalImagesPullingTime="2.597892577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:26.039740926 +0000 UTC m=+39.750412275" watchObservedRunningTime="2026-09-11 11:52:26.040503858 +0000 UTC m=+39.751175187"
Sep 11 11:52:28.437400 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.437362    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:28.437755 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:28.437525    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:28.437755 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:28.437591    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls podName:2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:36.437575519 +0000 UTC m=+50.148246835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls") pod "dns-default-ss9wb" (UID: "2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:28.843268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.843182    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"]
Sep 11 11:52:28.865190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.865169    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"]
Sep 11 11:52:28.865320 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.865298    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:28.868216 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.868191    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 11:52:28.868349 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.868260    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-59xvf\""
Sep 11 11:52:28.868349 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.868272    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 11:52:28.868523 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.868506    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:52:28.879141 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.879125    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"]
Sep 11 11:52:28.879227 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.879151    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"]
Sep 11 11:52:28.879264 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.879235    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:28.881784 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.881767    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 11:52:28.881908 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.881891    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 11:52:28.882040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.882025    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:52:28.882088 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.882045    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-78l97\""
Sep 11 11:52:28.882125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.882098    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 11:52:28.943146 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.943121    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f585bfd8b-vmzws"]
Sep 11 11:52:28.963260 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.963243    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f585bfd8b-vmzws"]
Sep 11 11:52:28.963366 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.963349    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:28.966182 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.966164    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 11:52:28.966286 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.966269    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 11:52:28.966350 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.966268    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 11:52:28.966403 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.966350    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 11:52:28.966403 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.966361    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 11:52:28.966516 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.966431    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 11:52:28.966680 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:28.966665    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ntzrd\""
Sep 11 11:52:29.041634 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.041609    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlhss\" (UniqueName: \"kubernetes.io/projected/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-kube-api-access-jlhss\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.041732 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.041643    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-config\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.041732 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.041663    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:29.041732 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.041688    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-serving-cert\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.041832 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.041766    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mzhn\" (UniqueName: \"kubernetes.io/projected/0e2694c2-21ad-4a31-8ff8-394583f36fb9-kube-api-access-2mzhn\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:29.044236 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.044218    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"]
Sep 11 11:52:29.067110 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.067093    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"]
Sep 11 11:52:29.067197 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.067133    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"
Sep 11 11:52:29.069898 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.069882    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wq5d5\""
Sep 11 11:52:29.090668 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.090642    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:29.142185 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142119    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.142185 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142159    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-stats-auth\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.142185 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142178    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.142345 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142209    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mzhn\" (UniqueName: \"kubernetes.io/projected/0e2694c2-21ad-4a31-8ff8-394583f36fb9-kube-api-access-2mzhn\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:29.142345 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142326    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlhss\" (UniqueName: \"kubernetes.io/projected/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-kube-api-access-jlhss\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.142458 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142349    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-default-certificate\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.142458 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142371    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-config\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.142458 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142396    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:29.142458 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142419    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-serving-cert\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.142633 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142466    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h4nq\" (UniqueName: \"kubernetes.io/projected/8a934645-42a5-466a-928f-acfe9ba67c9a-kube-api-access-5h4nq\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.142633 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.142528    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 11:52:29.142633 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.142578    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls podName:0e2694c2-21ad-4a31-8ff8-394583f36fb9 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:29.642564029 +0000 UTC m=+43.353235344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2xx52" (UID: "0e2694c2-21ad-4a31-8ff8-394583f36fb9") : secret "samples-operator-tls" not found
Sep 11 11:52:29.142954 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.142932    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-config\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.145818 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.145801    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-serving-cert\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.151733 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.151710    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mzhn\" (UniqueName: \"kubernetes.io/projected/0e2694c2-21ad-4a31-8ff8-394583f36fb9-kube-api-access-2mzhn\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:29.151815 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.151738    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlhss\" (UniqueName: \"kubernetes.io/projected/1b97718f-6b7f-4b58-b64f-dc1ebed0ce34-kube-api-access-jlhss\") pod \"service-ca-operator-cf7c776b7-gnh4f\" (UID: \"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.178297 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.178269    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:29.187088 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.187067    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"
Sep 11 11:52:29.234169 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.234140    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:29.243709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.243686    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.243814 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.243721    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-stats-auth\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.243814 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.243745    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.243931 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.243814    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-default-certificate\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.243931 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.243869    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:29.743845525 +0000 UTC m=+43.454516861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : configmap references non-existent config key: service-ca.crt
Sep 11 11:52:29.243931 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.243896    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:52:29.244086 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.243938    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zjnn\" (UniqueName: \"kubernetes.io/projected/a735687d-8f89-42de-b9c7-706218993b49-kube-api-access-7zjnn\") pod \"network-check-source-6f8b768455-r6gft\" (UID: \"a735687d-8f89-42de-b9c7-706218993b49\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"
Sep 11 11:52:29.244086 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.243952    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:29.743939433 +0000 UTC m=+43.454610764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : secret "router-metrics-certs-default" not found
Sep 11 11:52:29.244086 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.243989    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h4nq\" (UniqueName: \"kubernetes.io/projected/8a934645-42a5-466a-928f-acfe9ba67c9a-kube-api-access-5h4nq\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.246714 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.246348    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-stats-auth\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.246714 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.246409    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-default-certificate\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.253084 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.253066    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h4nq\" (UniqueName: \"kubernetes.io/projected/8a934645-42a5-466a-928f-acfe9ba67c9a-kube-api-access-5h4nq\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.344450 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.344418    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zjnn\" (UniqueName: \"kubernetes.io/projected/a735687d-8f89-42de-b9c7-706218993b49-kube-api-access-7zjnn\") pod \"network-check-source-6f8b768455-r6gft\" (UID: \"a735687d-8f89-42de-b9c7-706218993b49\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"
Sep 11 11:52:29.353343 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.353320    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zjnn\" (UniqueName: \"kubernetes.io/projected/a735687d-8f89-42de-b9c7-706218993b49-kube-api-access-7zjnn\") pod \"network-check-source-6f8b768455-r6gft\" (UID: \"a735687d-8f89-42de-b9c7-706218993b49\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"
Sep 11 11:52:29.375229 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.375208    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"
Sep 11 11:52:29.478188 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.478154    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f"]
Sep 11 11:52:29.647134 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.647100    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:29.647326 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.647260    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 11:52:29.647326 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.647324    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls podName:0e2694c2-21ad-4a31-8ff8-394583f36fb9 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:30.647308562 +0000 UTC m=+44.357979879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2xx52" (UID: "0e2694c2-21ad-4a31-8ff8-394583f36fb9") : secret "samples-operator-tls" not found
Sep 11 11:52:29.668614 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.668544    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-r6gft"]
Sep 11 11:52:29.671399 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:29.671375    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda735687d_8f89_42de_b9c7_706218993b49.slice/crio-890ca9801306b0627c57ca3a8fa07a51cbfb5a1beb51460bd1cfb91d2e0808b4 WatchSource:0}: Error finding container 890ca9801306b0627c57ca3a8fa07a51cbfb5a1beb51460bd1cfb91d2e0808b4: Status 404 returned error can't find the container with id 890ca9801306b0627c57ca3a8fa07a51cbfb5a1beb51460bd1cfb91d2e0808b4
Sep 11 11:52:29.747763 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.747728    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.747897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:29.747771    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:29.747976 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.747910    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:52:29.747976 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.747930    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:30.747907477 +0000 UTC m=+44.458578795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : configmap references non-existent config key: service-ca.crt
Sep 11 11:52:29.747976 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:29.747960    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:30.747947442 +0000 UTC m=+44.458618759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : secret "router-metrics-certs-default" not found
Sep 11 11:52:30.032424 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:30.032350    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft" event={"ID":"a735687d-8f89-42de-b9c7-706218993b49","Type":"ContainerStarted","Data":"890ca9801306b0627c57ca3a8fa07a51cbfb5a1beb51460bd1cfb91d2e0808b4"}
Sep 11 11:52:30.033475 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:30.033453    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f" event={"ID":"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34","Type":"ContainerStarted","Data":"5bed3f717eb62fc87f8df9add266873b2b6451ccfcda9c6e42ccc5f6939a300c"}
Sep 11 11:52:30.654977 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:30.654936    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:30.655377 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:30.655120    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 11:52:30.655377 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:30.655196    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls podName:0e2694c2-21ad-4a31-8ff8-394583f36fb9 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:32.655175749 +0000 UTC m=+46.365847080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2xx52" (UID: "0e2694c2-21ad-4a31-8ff8-394583f36fb9") : secret "samples-operator-tls" not found
Sep 11 11:52:30.756401 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:30.756364    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:30.756401 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:30.756417    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:30.756710 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:30.756555    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:52:30.756710 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:30.756598    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:32.756570839 +0000 UTC m=+46.467242184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : configmap references non-existent config key: service-ca.crt
Sep 11 11:52:30.756890 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:30.756743    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:32.756722737 +0000 UTC m=+46.467394055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : secret "router-metrics-certs-default" not found
Sep 11 11:52:32.173972 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:32.173930    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:32.264819 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:32.264723    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:32.359163 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:32.359114    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:32.415407 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:32.415373    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:32.674234 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:32.674191    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:32.674405 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:32.674346    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 11:52:32.674460 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:32.674417    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls podName:0e2694c2-21ad-4a31-8ff8-394583f36fb9 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:36.674396081 +0000 UTC m=+50.385067411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2xx52" (UID: "0e2694c2-21ad-4a31-8ff8-394583f36fb9") : secret "samples-operator-tls" not found
Sep 11 11:52:32.774983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:32.774950    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:32.774983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:32.774990    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:32.775206 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:32.775144    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:36.775121002 +0000 UTC m=+50.485794936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : configmap references non-existent config key: service-ca.crt
Sep 11 11:52:32.775206 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:32.775177    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:52:32.775310 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:32.775227    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:36.775216205 +0000 UTC m=+50.485887521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : secret "router-metrics-certs-default" not found
Sep 11 11:52:33.431315 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.431281    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:33.434740 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.434687    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pxlwr"]
Sep 11 11:52:33.450316 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.450206    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pxlwr"]
Sep 11 11:52:33.450426 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.450323    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.453036 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.453012    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 11:52:33.453144 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.453063    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 11:52:33.453253 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.453237    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 11:52:33.453397 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.453379    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 11:52:33.454479 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.454460    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-685qw\""
Sep 11 11:52:33.526895 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.526872    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:33.582275 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.582253    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22381901-1722-4032-888a-6ac1a3868149-crio-socket\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.582364 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.582296    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntjbl\" (UniqueName: \"kubernetes.io/projected/22381901-1722-4032-888a-6ac1a3868149-kube-api-access-ntjbl\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.582443 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.582418    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.582564 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.582548    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.582637 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.582622    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22381901-1722-4032-888a-6ac1a3868149-data-volume\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.662990 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.662960    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:33.683128 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.683088    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.683251 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.683143    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22381901-1722-4032-888a-6ac1a3868149-data-volume\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.683251 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.683167    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22381901-1722-4032-888a-6ac1a3868149-crio-socket\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.683251 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.683200    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntjbl\" (UniqueName: \"kubernetes.io/projected/22381901-1722-4032-888a-6ac1a3868149-kube-api-access-ntjbl\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.683440 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:33.683269    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:33.683440 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:33.683321    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:33.683440 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:33.683333    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:34.183311839 +0000 UTC m=+47.893983170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:33.683440 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:33.683364    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:34.18335052 +0000 UTC m=+47.894021856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:33.683440 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.683427    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22381901-1722-4032-888a-6ac1a3868149-crio-socket\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.683737 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.683270    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.683737 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.683527    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22381901-1722-4032-888a-6ac1a3868149-data-volume\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.697211 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.697153    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntjbl\" (UniqueName: \"kubernetes.io/projected/22381901-1722-4032-888a-6ac1a3868149-kube-api-access-ntjbl\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:33.725396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.725371    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:33.826241 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:33.826205    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:34.049128 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.049021    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft" event={"ID":"a735687d-8f89-42de-b9c7-706218993b49","Type":"ContainerStarted","Data":"03c696da76ce79506800c234257d1da577ffda3a743cb24f9496bfbca031f1cb"}
Sep 11 11:52:34.053665 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.053632    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f" event={"ID":"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34","Type":"ContainerStarted","Data":"6cdb71b3d605d0ba6c146b470f9165f761dff8e23b0733c98485267494ed477c"}
Sep 11 11:52:34.065539 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.065460    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r6gft" podStartSLOduration=1.238981481 podStartE2EDuration="5.065443461s" totalImagesPullingTime="3.82646198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:34.064290696 +0000 UTC m=+47.774962030" watchObservedRunningTime="2026-09-11 11:52:34.065443461 +0000 UTC m=+47.776114801"
Sep 11 11:52:34.082371 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.082325    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f" podStartSLOduration=2.3789991600000002 podStartE2EDuration="6.0823073s" totalImagesPullingTime="3.70330814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:34.081806274 +0000 UTC m=+47.792477613" watchObservedRunningTime="2026-09-11 11:52:34.0823073 +0000 UTC m=+47.792978640"
Sep 11 11:52:34.187875 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.187839    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:34.188024 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:34.188006    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:34.188319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.188083    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:34.188319 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:34.188093    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:35.188073018 +0000 UTC m=+48.898744341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:34.188319 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:34.188202    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:34.188319 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:34.188283    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:35.188259247 +0000 UTC m=+48.898930580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:34.721324 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.721274    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:34.781447 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:34.781401    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:35.197106 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:35.197074    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:35.197297 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:35.197185    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:35.197297 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:35.197192    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:35.197297 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:35.197272    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.197239754 +0000 UTC m=+50.907911086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:35.197468 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:35.197324    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:35.197468 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:35.197375    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.197362763 +0000 UTC m=+50.908034083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:35.393714 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:35.393690    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzkmp_21124859-d604-4df1-b825-d6dbfb5ecace/dns-node-resolver/0.log"
Sep 11 11:52:35.414889 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:35.414855    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:35.513020 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:35.512961    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:36.427628 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:36.427582    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:36.479330 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:36.479303    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:36.508306 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:36.508281    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:36.508424 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:36.508408    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:36.508476 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:36.508465    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls podName:2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:52.508450641 +0000 UTC m=+66.219121956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls") pod "dns-default-ss9wb" (UID: "2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:36.592158 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:36.592138    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qqt5h_868bf557-a754-4bc4-92bb-54bf847ab8f7/node-ca/0.log"
Sep 11 11:52:36.710685 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:36.710629    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:36.710787 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:36.710753    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 11:52:36.710824 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:36.710801    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls podName:0e2694c2-21ad-4a31-8ff8-394583f36fb9 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:44.710789396 +0000 UTC m=+58.421460725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2xx52" (UID: "0e2694c2-21ad-4a31-8ff8-394583f36fb9") : secret "samples-operator-tls" not found
Sep 11 11:52:36.811501 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:36.811452    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:36.811683 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:36.811511    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:36.811683 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:36.811600    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:44.811584251 +0000 UTC m=+58.522255591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : configmap references non-existent config key: service-ca.crt
Sep 11 11:52:36.811683 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:36.811656    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:52:36.811807 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:36.811712    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:52:44.811700232 +0000 UTC m=+58.522371548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : secret "router-metrics-certs-default" not found
Sep 11 11:52:37.053044 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:37.053005    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:37.153936 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:37.153902    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:37.214141 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:37.214111    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:37.214285 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:37.214187    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:37.214285 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:37.214252    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:37.214414 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:37.214334    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:41.214309886 +0000 UTC m=+54.924981222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:37.214414 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:37.214337    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:37.214414 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:37.214383    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:41.214367016 +0000 UTC m=+54.925038347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:37.393122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:37.393025    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l68qh_71cca530-f27e-4d2a-8a03-e6a7d99064a5/serve-healthcheck-canary/0.log"
Sep 11 11:52:38.801659 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:38.801622    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:40.564077 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:40.564031    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:41.243425 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:41.243391    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:41.243620 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:41.243501    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:41.243620 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:41.243534    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:41.243620 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:41.243608    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:49.243587276 +0000 UTC m=+62.954258606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:41.243620 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:41.243610    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:41.243773 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:41.243650    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:49.243638872 +0000 UTC m=+62.954310189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:42.195528 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:42.195472    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:42.417045 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:42.417006    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:42.517805 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:42.517730    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:42.619351 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:42.619319    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:42.774463 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:42.774388    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:42.874777 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:42.874746    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:43.459105 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:43.459069    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:43.514239 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:43.514207    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:43.890230 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:43.890196    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:44.684060 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.684032    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:44.742670 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.742645    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:44.771767 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.771747    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:44.774217 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.774200    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e2694c2-21ad-4a31-8ff8-394583f36fb9-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2xx52\" (UID: \"0e2694c2-21ad-4a31-8ff8-394583f36fb9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:44.774307 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.774266    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"
Sep 11 11:52:44.872770 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.872740    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:44.872899 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.872777    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:52:44.872942 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:44.872924    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 11:52:44.872986 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:44.872966    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:53:00.872940213 +0000 UTC m=+74.583611551 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : configmap references non-existent config key: service-ca.crt
Sep 11 11:52:44.873027 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:44.872999    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs podName:8a934645-42a5-466a-928f-acfe9ba67c9a nodeName:}" failed. No retries permitted until 2026-09-11 11:53:00.87298838 +0000 UTC m=+74.583659699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs") pod "router-default-7f585bfd8b-vmzws" (UID: "8a934645-42a5-466a-928f-acfe9ba67c9a") : secret "router-metrics-certs-default" not found
Sep 11 11:52:44.892256 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:44.892232    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:45.070011 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:45.069946    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52"]
Sep 11 11:52:45.091378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:45.091350    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:45.487558 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:45.487461    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:45.882279 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:45.882242    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:46.077652 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:46.077618    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:46.084179 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:46.084154    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52" event={"ID":"0e2694c2-21ad-4a31-8ff8-394583f36fb9","Type":"ContainerStarted","Data":"63c6c47835d0e1d1830001d7e631a197afd9f6622cac31762b5df185d74ae66e"}
Sep 11 11:52:46.289969 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:46.289942    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:46.580142 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:46.580058    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:46.689365 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:46.689327    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:46.891570 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:46.891495    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:47.210547 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:47.210453    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:47.311701 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:47.311673    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:47.362390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:47.362360    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.012174 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.012149    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92cnc"
Sep 11 11:52:48.058369 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.058331    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.092471 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.092442    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52" event={"ID":"0e2694c2-21ad-4a31-8ff8-394583f36fb9","Type":"ContainerStarted","Data":"21e5801a222f2cd1e5da5f609c4b4a26dd6b3914b4af384cbe8f52923a64acae"}
Sep 11 11:52:48.092683 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.092519    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52" event={"ID":"0e2694c2-21ad-4a31-8ff8-394583f36fb9","Type":"ContainerStarted","Data":"43b7e687d2f7f0f80fc8bbd57f9e70a4fa423db9280fcb3ab568bb06667304f6"}
Sep 11 11:52:48.108903 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.108861    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2xx52" podStartSLOduration=17.309188371 podStartE2EDuration="20.108848069s" totalImagesPullingTime="2.799659698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:48.108590837 +0000 UTC m=+61.819262177" watchObservedRunningTime="2026-09-11 11:52:48.108848069 +0000 UTC m=+61.819519425"
Sep 11 11:52:48.290870 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.290839    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.490672 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.490637    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.545940 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.545865    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.689356 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.689326    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.888313 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:48.888244    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:49.086178 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:49.086143    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:49.307547 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:49.307523    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:49.307713 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:49.307591    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:49.307713 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:49.307672    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:49.307829 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:52:49.307730    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls podName:22381901-1722-4032-888a-6ac1a3868149 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:05.307715718 +0000 UTC m=+79.018387034 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pxlwr" (UID: "22381901-1722-4032-888a-6ac1a3868149") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:49.308072 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:49.308050    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22381901-1722-4032-888a-6ac1a3868149-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:52:49.493505 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:49.493451    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:50.291575 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:50.291544    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:50.691250 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:50.691160    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:50.882819 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:50.882776    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:51.086602 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.086568    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:51.285419 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.285388    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:51.525525 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.525426    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:51.528689 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.528670    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:52:51.538456 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.538437    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfb47d93-fe8c-42eb-a7f7-0897d42c72ff-metrics-certs\") pod \"network-metrics-daemon-v66kj\" (UID: \"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff\") " pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:51.625944 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.625914    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:51.628673 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.628655    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:52:51.639274 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.639247    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49f88faf-4689-4ca0-a18a-4cb7d895eea8-original-pull-secret\") pod \"global-pull-secret-syncer-xq77w\" (UID: \"49f88faf-4689-4ca0-a18a-4cb7d895eea8\") " pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:51.726455 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.726431    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:51.728494 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.728471    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq77w"
Sep 11 11:52:51.728878 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.728860    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m87mk\" (UniqueName: \"kubernetes.io/projected/42905634-6c88-439a-9b27-ccf67a851214-kube-api-access-m87mk\") pod \"network-check-target-fndk9\" (UID: \"42905634-6c88-439a-9b27-ccf67a851214\") " pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:51.832060 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.832039    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f8bwk\""
Sep 11 11:52:51.837942 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.837926    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-74fqm\""
Sep 11 11:52:51.840051 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.840036    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:51.846304 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:51.846288    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v66kj"
Sep 11 11:52:52.022305 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.022270    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xq77w"]
Sep 11 11:52:52.025516 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:52.025453    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f88faf_4689_4ca0_a18a_4cb7d895eea8.slice/crio-3dbba202a7fbe22bd4cc621279fb69e9bb113556b91121a504ee41246aa5106f WatchSource:0}: Error finding container 3dbba202a7fbe22bd4cc621279fb69e9bb113556b91121a504ee41246aa5106f: Status 404 returned error can't find the container with id 3dbba202a7fbe22bd4cc621279fb69e9bb113556b91121a504ee41246aa5106f
Sep 11 11:52:52.105578 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.105479    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xq77w" event={"ID":"49f88faf-4689-4ca0-a18a-4cb7d895eea8","Type":"ContainerStarted","Data":"3dbba202a7fbe22bd4cc621279fb69e9bb113556b91121a504ee41246aa5106f"}
Sep 11 11:52:52.120052 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.119995    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fndk9"]
Sep 11 11:52:52.122395 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:52.122373    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42905634_6c88_439a_9b27_ccf67a851214.slice/crio-bd2689cd40420dec2c2b61897c57d3eeeb0cae255d5190f4cc2edadf1ae0c62b WatchSource:0}: Error finding container bd2689cd40420dec2c2b61897c57d3eeeb0cae255d5190f4cc2edadf1ae0c62b: Status 404 returned error can't find the container with id bd2689cd40420dec2c2b61897c57d3eeeb0cae255d5190f4cc2edadf1ae0c62b
Sep 11 11:52:52.147589 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.147568    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v66kj"]
Sep 11 11:52:52.149941 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:52.149919    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb47d93_fe8c_42eb_a7f7_0897d42c72ff.slice/crio-0849e875fd3ed92ac5d4114e42970172d8f8758f3f8339d7129729541d65ffc7 WatchSource:0}: Error finding container 0849e875fd3ed92ac5d4114e42970172d8f8758f3f8339d7129729541d65ffc7: Status 404 returned error can't find the container with id 0849e875fd3ed92ac5d4114e42970172d8f8758f3f8339d7129729541d65ffc7
Sep 11 11:52:52.288839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.288808    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:52.483913 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.483854    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:52.532984 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.532958    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:52.535620 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.535602    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a-metrics-tls\") pod \"dns-default-ss9wb\" (UID: \"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a\") " pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:52.683356 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.683323    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:52.742853 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.742787    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mvnq6\""
Sep 11 11:52:52.751468 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.751281    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:52.784003 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.783942    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:52.885773 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.885714    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:52.970453 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:52.970425    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:53.047841 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.047813    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ss9wb"]
Sep 11 11:52:53.051725 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:53.051694    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd6e280_bc22_4dc4_bf7c_47a8adb34b4a.slice/crio-eb5698ab18fe839a9e5d732135cf70a95316ad6c644e92092b22afa1623573af WatchSource:0}: Error finding container eb5698ab18fe839a9e5d732135cf70a95316ad6c644e92092b22afa1623573af: Status 404 returned error can't find the container with id eb5698ab18fe839a9e5d732135cf70a95316ad6c644e92092b22afa1623573af
Sep 11 11:52:53.074246 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.074220    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:53.110588 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.110555    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v66kj" event={"ID":"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff","Type":"ContainerStarted","Data":"0849e875fd3ed92ac5d4114e42970172d8f8758f3f8339d7129729541d65ffc7"}
Sep 11 11:52:53.112068 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.111984    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ss9wb" event={"ID":"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a","Type":"ContainerStarted","Data":"eb5698ab18fe839a9e5d732135cf70a95316ad6c644e92092b22afa1623573af"}
Sep 11 11:52:53.114076 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.114041    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fndk9" event={"ID":"42905634-6c88-439a-9b27-ccf67a851214","Type":"ContainerStarted","Data":"d99e268e583b3e2c45e9b95f007c4542f27fb0b7727a5c1cb8ced179cfd0c6fd"}
Sep 11 11:52:53.114173 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.114089    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fndk9" event={"ID":"42905634-6c88-439a-9b27-ccf67a851214","Type":"ContainerStarted","Data":"bd2689cd40420dec2c2b61897c57d3eeeb0cae255d5190f4cc2edadf1ae0c62b"}
Sep 11 11:52:53.114344 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.114323    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:52:53.131289 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.131236    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fndk9" podStartSLOduration=66.131217438 podStartE2EDuration="1m6.131217438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:53.130740086 +0000 UTC m=+66.841411427" watchObservedRunningTime="2026-09-11 11:52:53.131217438 +0000 UTC m=+66.841888777"
Sep 11 11:52:53.448238 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.448200    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:53.549045 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.548985    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:53.881093 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.881056    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:53.981255 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:53.981221    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:54.123780 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:54.123747    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v66kj" event={"ID":"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff","Type":"ContainerStarted","Data":"2dbd1a51b7751008f4f4cf219b9b3cfac94a230ce92c136a3672349c3fda7a35"}
Sep 11 11:52:54.123947 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:54.123804    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v66kj" event={"ID":"dfb47d93-fe8c-42eb-a7f7-0897d42c72ff","Type":"ContainerStarted","Data":"edb363230ace7fb4ecbdaa56d9beece5d6fdc15e483895dc6853a113777ef363"}
Sep 11 11:52:54.141254 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:54.141110    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v66kj" podStartSLOduration=66.93957084 podStartE2EDuration="1m8.141088261s" totalImagesPullingTime="1.201517421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:54.139321277 +0000 UTC m=+67.849992618" watchObservedRunningTime="2026-09-11 11:52:54.141088261 +0000 UTC m=+67.851759599"
Sep 11 11:52:54.277785 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:54.277751    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:54.665703 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:54.665668    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:54.765028 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:54.764994    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:54.873827 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:54.873792    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:55.494791 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.494762    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"]
Sep 11 11:52:55.498127 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.498108    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.500942 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.500915    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 11:52:55.502404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.502288    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 11 11:52:55.502404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.502306    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 11:52:55.502404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.502315    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 11:52:55.509026 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.508974    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"]
Sep 11 11:52:55.558124 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.558098    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-klusterlet-config\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.558280 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.558151    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88wdb\" (UniqueName: \"kubernetes.io/projected/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-kube-api-access-88wdb\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.558280 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.558193    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-tmp\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.594759 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.594719    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-6dwvk"]
Sep 11 11:52:55.598071 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.598046    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-6dwvk"
Sep 11 11:52:55.601318 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.601298    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vtvss\""
Sep 11 11:52:55.601659 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.601643    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 11:52:55.601754 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.601643    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 11:52:55.610335 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.610268    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:55.611045 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.611025    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-6dwvk"]
Sep 11 11:52:55.659249 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.659217    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-klusterlet-config\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.659514 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.659281    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88wdb\" (UniqueName: \"kubernetes.io/projected/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-kube-api-access-88wdb\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.659514 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.659313    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26frz\" (UniqueName: \"kubernetes.io/projected/fb40254a-26c2-495e-bc2c-48f1d22ed985-kube-api-access-26frz\") pod \"downloads-76f99584f4-6dwvk\" (UID: \"fb40254a-26c2-495e-bc2c-48f1d22ed985\") " pod="openshift-console/downloads-76f99584f4-6dwvk"
Sep 11 11:52:55.659514 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.659354    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-tmp\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.659786 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.659754    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-tmp\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.662129 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.662105    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-klusterlet-config\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.668832 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.668813    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88wdb\" (UniqueName: \"kubernetes.io/projected/909beb9d-3050-4dfa-bf6b-41e6d7f01ddf-kube-api-access-88wdb\") pod \"klusterlet-addon-workmgr-548579cd7d-h2bv9\" (UID: \"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.760087 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.760003    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26frz\" (UniqueName: \"kubernetes.io/projected/fb40254a-26c2-495e-bc2c-48f1d22ed985-kube-api-access-26frz\") pod \"downloads-76f99584f4-6dwvk\" (UID: \"fb40254a-26c2-495e-bc2c-48f1d22ed985\") " pod="openshift-console/downloads-76f99584f4-6dwvk"
Sep 11 11:52:55.769730 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.769707    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26frz\" (UniqueName: \"kubernetes.io/projected/fb40254a-26c2-495e-bc2c-48f1d22ed985-kube-api-access-26frz\") pod \"downloads-76f99584f4-6dwvk\" (UID: \"fb40254a-26c2-495e-bc2c-48f1d22ed985\") " pod="openshift-console/downloads-76f99584f4-6dwvk"
Sep 11 11:52:55.809739 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.809713    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:52:55.898393 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.898359    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:55.909825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.909799    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-6dwvk"
Sep 11 11:52:55.998278 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:55.998243    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:56.104405 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:56.104324    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:56.493988 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:56.493957    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-6dwvk"]
Sep 11 11:52:56.496346 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:56.496316    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"]
Sep 11 11:52:56.499694 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:56.499667    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb40254a_26c2_495e_bc2c_48f1d22ed985.slice/crio-378f90e4a556f01432e9a072d915eea615e7bf5a2eb6060a5fc9f8100379a906 WatchSource:0}: Error finding container 378f90e4a556f01432e9a072d915eea615e7bf5a2eb6060a5fc9f8100379a906: Status 404 returned error can't find the container with id 378f90e4a556f01432e9a072d915eea615e7bf5a2eb6060a5fc9f8100379a906
Sep 11 11:52:56.500579 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:52:56.500549    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909beb9d_3050_4dfa_bf6b_41e6d7f01ddf.slice/crio-ed2e45085295e7c0ed5424427e13474b594f788825b58d2424e41cf61ce80c14 WatchSource:0}: Error finding container ed2e45085295e7c0ed5424427e13474b594f788825b58d2424e41cf61ce80c14: Status 404 returned error can't find the container with id ed2e45085295e7c0ed5424427e13474b594f788825b58d2424e41cf61ce80c14
Sep 11 11:52:57.138801 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.138766    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xq77w" event={"ID":"49f88faf-4689-4ca0-a18a-4cb7d895eea8","Type":"ContainerStarted","Data":"2c0cab3f33901da4c6656d4002cd17a086833692098ba00bfa692bd66eaf0650"}
Sep 11 11:52:57.140615 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.140594    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ss9wb" event={"ID":"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a","Type":"ContainerStarted","Data":"2ac7d62da3a2725a7c4582a53fa1397674b68d6797ddf9583a90b301a13fbcca"}
Sep 11 11:52:57.140725 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.140633    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ss9wb" event={"ID":"2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a","Type":"ContainerStarted","Data":"ec70cc170f6310eaa50284a0b56a30058bd1dcff7e5208044a7ac23328f869bb"}
Sep 11 11:52:57.140725 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.140675    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:52:57.141879 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.141860    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-6dwvk" event={"ID":"fb40254a-26c2-495e-bc2c-48f1d22ed985","Type":"ContainerStarted","Data":"378f90e4a556f01432e9a072d915eea615e7bf5a2eb6060a5fc9f8100379a906"}
Sep 11 11:52:57.142850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.142828    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9" event={"ID":"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf","Type":"ContainerStarted","Data":"ed2e45085295e7c0ed5424427e13474b594f788825b58d2424e41cf61ce80c14"}
Sep 11 11:52:57.153455 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.153317    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xq77w" podStartSLOduration=65.957908667 podStartE2EDuration="1m10.153302167s" totalImagesPullingTime="4.1953935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:57.152909168 +0000 UTC m=+70.863580531" watchObservedRunningTime="2026-09-11 11:52:57.153302167 +0000 UTC m=+70.863973506"
Sep 11 11:52:57.169334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:52:57.169285    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ss9wb" podStartSLOduration=34.015313149 podStartE2EDuration="37.169263838s" totalImagesPullingTime="3.153950689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:57.168623511 +0000 UTC m=+70.879294850" watchObservedRunningTime="2026-09-11 11:52:57.169263838 +0000 UTC m=+70.879935176"
Sep 11 11:53:00.905634 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:00.905596    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:00.906040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:00.905653    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:00.906306 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:00.906260    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a934645-42a5-466a-928f-acfe9ba67c9a-service-ca-bundle\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:00.908731 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:00.908678    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a934645-42a5-466a-928f-acfe9ba67c9a-metrics-certs\") pod \"router-default-7f585bfd8b-vmzws\" (UID: \"8a934645-42a5-466a-928f-acfe9ba67c9a\") " pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:01.075037 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:01.075005    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ntzrd\""
Sep 11 11:53:01.082696 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:01.082454    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:01.381429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:01.381395    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f585bfd8b-vmzws"]
Sep 11 11:53:01.384727 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:01.384696    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a934645_42a5_466a_928f_acfe9ba67c9a.slice/crio-53620e6ffab3f82bfa0facf3fb41fd8dd946b6a5ecd23c7ace998c72e1d39cef WatchSource:0}: Error finding container 53620e6ffab3f82bfa0facf3fb41fd8dd946b6a5ecd23c7ace998c72e1d39cef: Status 404 returned error can't find the container with id 53620e6ffab3f82bfa0facf3fb41fd8dd946b6a5ecd23c7ace998c72e1d39cef
Sep 11 11:53:02.168011 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:02.167956    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9" event={"ID":"909beb9d-3050-4dfa-bf6b-41e6d7f01ddf","Type":"ContainerStarted","Data":"7b8cfd1cd0cc19b81f32a108c6f8ad2cf5aecd05c5796e877fd37b55b63b5f0f"}
Sep 11 11:53:02.168460 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:02.168066    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:53:02.170021 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:02.169996    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f585bfd8b-vmzws" event={"ID":"8a934645-42a5-466a-928f-acfe9ba67c9a","Type":"ContainerStarted","Data":"4c483cc3fc1b2ac6c8d1cc14346b086a569925baa337baa5cc971a7558e7f8d1"}
Sep 11 11:53:02.170131 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:02.170035    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f585bfd8b-vmzws" event={"ID":"8a934645-42a5-466a-928f-acfe9ba67c9a","Type":"ContainerStarted","Data":"53620e6ffab3f82bfa0facf3fb41fd8dd946b6a5ecd23c7ace998c72e1d39cef"}
Sep 11 11:53:02.170131 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:02.170125    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9"
Sep 11 11:53:02.183851 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:02.183804    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548579cd7d-h2bv9" podStartSLOduration=2.499566527 podStartE2EDuration="7.18378959s" totalImagesPullingTime="4.684223063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:02.182907828 +0000 UTC m=+75.893579181" watchObservedRunningTime="2026-09-11 11:53:02.18378959 +0000 UTC m=+75.894460928"
Sep 11 11:53:02.214927 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:02.214871    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f585bfd8b-vmzws" podStartSLOduration=34.214852348 podStartE2EDuration="34.214852348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:02.213551176 +0000 UTC m=+75.924222514" watchObservedRunningTime="2026-09-11 11:53:02.214852348 +0000 UTC m=+75.925523687"
Sep 11 11:53:03.083443 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:03.083410    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:03.086297 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:03.086271    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:03.173668 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:03.173616    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:03.174793 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:03.174773    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f585bfd8b-vmzws"
Sep 11 11:53:03.539874 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:03.539849    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:03.588284 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:03.588200    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:03.917549 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:03.917452    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:04.017934 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:04.017887    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:04.087415 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:04.087337    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:04.271278 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:04.271201    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:05.339349 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:05.339318    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:53:05.342001 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:05.341977    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22381901-1722-4032-888a-6ac1a3868149-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pxlwr\" (UID: \"22381901-1722-4032-888a-6ac1a3868149\") " pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:53:05.562420 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:05.562386    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-685qw\""
Sep 11 11:53:05.569765 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:05.569742    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pxlwr"
Sep 11 11:53:05.873140 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:05.873061    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pxlwr"]
Sep 11 11:53:05.876795 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:05.876763    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22381901_1722_4032_888a_6ac1a3868149.slice/crio-0ef5c31c86b3011854cbce87db7b36a436da65e7d8f8701364bc364d85b971cd WatchSource:0}: Error finding container 0ef5c31c86b3011854cbce87db7b36a436da65e7d8f8701364bc364d85b971cd: Status 404 returned error can't find the container with id 0ef5c31c86b3011854cbce87db7b36a436da65e7d8f8701364bc364d85b971cd
Sep 11 11:53:06.185005 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.184924    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pxlwr" event={"ID":"22381901-1722-4032-888a-6ac1a3868149","Type":"ContainerStarted","Data":"2642d87c8dff1c6656bada93ac1923fac5fb37a011235a842a10c60a865a975a"}
Sep 11 11:53:06.185005 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.184991    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pxlwr" event={"ID":"22381901-1722-4032-888a-6ac1a3868149","Type":"ContainerStarted","Data":"0ef5c31c86b3011854cbce87db7b36a436da65e7d8f8701364bc364d85b971cd"}
Sep 11 11:53:06.500455 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.500380    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bd449b7f7-hbhw6"]
Sep 11 11:53:06.503612 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.503582    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.507116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.507035    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wrkjb\""
Sep 11 11:53:06.507239 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.507155    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 11:53:06.507239 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.507162    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 11:53:06.507404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.507383    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:06.508514 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.508477    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 11:53:06.508618 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.508512    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 11:53:06.508618 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.508530    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 11:53:06.515560 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.514584    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bd449b7f7-hbhw6"]
Sep 11 11:53:06.549849 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.549824    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-service-ca\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.549984 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.549868    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-serving-cert\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.549984 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.549899    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-oauth-serving-cert\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.549984 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.549921    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-config\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.550122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.549994    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-oauth-config\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.550122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.550022    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxtvz\" (UniqueName: \"kubernetes.io/projected/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-kube-api-access-wxtvz\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.560583 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.560543    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:06.651218 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.651178    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-oauth-config\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.651390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.651236    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxtvz\" (UniqueName: \"kubernetes.io/projected/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-kube-api-access-wxtvz\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.651390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.651281    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-service-ca\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.651390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.651309    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-serving-cert\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.651390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.651332    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-oauth-serving-cert\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.651390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.651364    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-config\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.652217 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.652124    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-service-ca\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.652217 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.652172    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-config\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.652217 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.652211    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-oauth-serving-cert\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.654069 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.654044    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-oauth-config\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.654199 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.654176    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-serving-cert\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.659729 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.659707    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxtvz\" (UniqueName: \"kubernetes.io/projected/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-kube-api-access-wxtvz\") pod \"console-5bd449b7f7-hbhw6\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") " pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:06.815309 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:06.815279    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:07.106805 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:07.106716    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:07.152925 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:07.152898    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ss9wb"
Sep 11 11:53:07.175358 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:07.175296    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:07.293610 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:07.293579    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:08.289522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.289385    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6f486dd7-7w98w"]
Sep 11 11:53:08.293800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.293779    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.304535 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.304511    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 11:53:08.305979 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.305956    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6f486dd7-7w98w"]
Sep 11 11:53:08.363618 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.363594    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-serving-cert\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.363735 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.363640    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw9fw\" (UniqueName: \"kubernetes.io/projected/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-kube-api-access-qw9fw\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.363735 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.363671    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-service-ca\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.363840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.363787    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-oauth-config\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.363889 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.363845    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-config\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.363889 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.363871    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-oauth-serving-cert\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.363982 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.363905    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-trusted-ca-bundle\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465084 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465056    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw9fw\" (UniqueName: \"kubernetes.io/projected/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-kube-api-access-qw9fw\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465221 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465100    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-service-ca\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465221 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465150    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-oauth-config\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465221 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465186    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-config\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465221 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465203    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-oauth-serving-cert\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465374 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465224    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-trusted-ca-bundle\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465374 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465253    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-serving-cert\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.465970 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.465945    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-config\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.466095 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.466008    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-oauth-serving-cert\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.466095 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.466046    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-service-ca\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.466355 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.466338    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-trusted-ca-bundle\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.468443 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.468385    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-oauth-config\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.468568 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.468464    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-serving-cert\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.473424 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.473393    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw9fw\" (UniqueName: \"kubernetes.io/projected/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-kube-api-access-qw9fw\") pod \"console-c6f486dd7-7w98w\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") " pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.533339 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.533308    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:08.605595 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.605529    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:08.633153 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:08.633028    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:09.453216 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:09.453182    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:10.679153 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:10.679115    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.237400 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.237361    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.339002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.338966    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.438933 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.438900    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.488835 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.488763    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"]
Sep 11 11:53:11.492357 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.492340    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.497146 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.497122    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 11:53:11.497261 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.497216    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hfgj2\""
Sep 11 11:53:11.497261 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.497233    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 11:53:11.497386 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.497362    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 11:53:11.497454 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.497433    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 11:53:11.497582 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.497570    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 11:53:11.497582 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.497571    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 11:53:11.503152 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.503131    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kxrzn"]
Sep 11 11:53:11.506529 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.506507    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"]
Sep 11 11:53:11.506640 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.506626    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.509344 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.509326    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 11:53:11.509672 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.509648    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 11:53:11.509983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.509650    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 11:53:11.509983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.509864    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gz4mc\""
Sep 11 11:53:11.518914 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.518893    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.587325 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.587294    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.589793 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589768    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-sys\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.589901 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589806    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-wtmp\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.589901 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589836    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.589901 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589863    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.589901 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589886    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71251815-9411-41cb-bb13-30c51f158ebe-metrics-client-ca\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589906    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac6fe090-abfb-42d6-995b-0ab2da0c0969-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589938    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78zch\" (UniqueName: \"kubernetes.io/projected/71251815-9411-41cb-bb13-30c51f158ebe-kube-api-access-78zch\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589954    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.589997    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-root\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.590022    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-tls\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.590039    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-accelerators-collector-config\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.590081    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-textfile\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.590116 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.590104    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.590418 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.590122    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf8df\" (UniqueName: \"kubernetes.io/projected/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-api-access-bf8df\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.590418 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.590142    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac6fe090-abfb-42d6-995b-0ab2da0c0969-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.687874 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.687847    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.690836 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.690809    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-textfile\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.690958 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.690842    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.690958 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.690870    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf8df\" (UniqueName: \"kubernetes.io/projected/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-api-access-bf8df\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.690958 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.690900    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac6fe090-abfb-42d6-995b-0ab2da0c0969-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.690958 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.690927    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-sys\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691167 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.690978    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-wtmp\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691167 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691008    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691167 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691037    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.691167 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691070    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71251815-9411-41cb-bb13-30c51f158ebe-metrics-client-ca\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691167 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691096    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac6fe090-abfb-42d6-995b-0ab2da0c0969-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.691167 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691128    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78zch\" (UniqueName: \"kubernetes.io/projected/71251815-9411-41cb-bb13-30c51f158ebe-kube-api-access-78zch\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691167 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691153    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.691508 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691190    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-root\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691508 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691217    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-tls\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691508 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691244    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-accelerators-collector-config\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691508 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691249    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-wtmp\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691734 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691541    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac6fe090-abfb-42d6-995b-0ab2da0c0969-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.691734 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691244    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-textfile\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691837 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691763    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.691894 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691839    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-root\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.691944 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691928    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac6fe090-abfb-42d6-995b-0ab2da0c0969-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.692020 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.691991    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71251815-9411-41cb-bb13-30c51f158ebe-sys\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.692088 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:53:11.692072    2586 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 11 11:53:11.692170 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:53:11.692157    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-tls podName:ac6fe090-abfb-42d6-995b-0ab2da0c0969 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:12.192136777 +0000 UTC m=+85.902808107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-zcdrs" (UID: "ac6fe090-abfb-42d6-995b-0ab2da0c0969") : secret "kube-state-metrics-tls" not found
Sep 11 11:53:11.692170 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.692159    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71251815-9411-41cb-bb13-30c51f158ebe-metrics-client-ca\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.692404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.692203    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-accelerators-collector-config\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.694405 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.694378    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.694539 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.694434    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-tls\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.694539 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.694509    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71251815-9411-41cb-bb13-30c51f158ebe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.700448 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.700426    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf8df\" (UniqueName: \"kubernetes.io/projected/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-api-access-bf8df\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:11.700580 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.700546    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78zch\" (UniqueName: \"kubernetes.io/projected/71251815-9411-41cb-bb13-30c51f158ebe-kube-api-access-78zch\") pod \"node-exporter-kxrzn\" (UID: \"71251815-9411-41cb-bb13-30c51f158ebe\") " pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.787401 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.787324    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.817651 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.817624    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kxrzn"
Sep 11 11:53:11.887137 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:11.887109    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:12.162885 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.162847    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:12.194759 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.194725    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:12.197587 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.197553    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac6fe090-abfb-42d6-995b-0ab2da0c0969-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-zcdrs\" (UID: \"ac6fe090-abfb-42d6-995b-0ab2da0c0969\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:12.403698 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.403665    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"
Sep 11 11:53:12.462614 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.462548    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:12.549291 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.549256    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:12.555265 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.555238    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:53:12.558998 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.558980    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.561866 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.561844    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n4lvm\""
Sep 11 11:53:12.561998 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.561845    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 11:53:12.562063 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.561853    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 11:53:12.562063 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.561852    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 11:53:12.562063 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.562048    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 11:53:12.562200 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.562182    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 11:53:12.562314 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.562292    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 11:53:12.562429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.562336    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 11:53:12.562578 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.562563    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 11:53:12.563603 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.563584    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 11:53:12.575387 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.575364    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:53:12.668434 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.668399    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:12.699544 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699517    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-web-config\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699567    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699597    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nztw\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-kube-api-access-6nztw\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699646    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699685    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699740    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699831    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699866    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.699923 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699902    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.700219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699947    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-out\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.700219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.699989    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.700219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.700014    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.700219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.700039    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.800942 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.800919    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801101 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.800958    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801101 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.800985    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-web-config\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801101 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801020    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801101 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801038    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nztw\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-kube-api-access-6nztw\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801101 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801057    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801101 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801083    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801112    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801132    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801147    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801175    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801196    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-out\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.801396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801215    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.802065 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.801997    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.803189 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.803152    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.803624 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.803433    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.804675 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.804653    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.805198 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.805127    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.805294 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.805196    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.805920 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.805881    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.806009 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.805977    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-out\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.806009 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.805981    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.806886 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.806867    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-web-config\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.806958 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.806900    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.806995 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.806959    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.810340 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.810320    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nztw\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-kube-api-access-6nztw\") pod \"alertmanager-main-0\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:12.869679 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:12.869655    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:53:13.524865 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:13.524827    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:13.599060 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:13.598688    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71251815_9411_41cb_bb13_30c51f158ebe.slice/crio-190c8e7f32408d8cfe9e391143433cd3db7cc0e8f1df286ff9a973e7ef316e9e WatchSource:0}: Error finding container 190c8e7f32408d8cfe9e391143433cd3db7cc0e8f1df286ff9a973e7ef316e9e: Status 404 returned error can't find the container with id 190c8e7f32408d8cfe9e391143433cd3db7cc0e8f1df286ff9a973e7ef316e9e
Sep 11 11:53:13.872571 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:13.872543    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs"]
Sep 11 11:53:13.874877 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:13.874853    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6fe090_abfb_42d6_995b_0ab2da0c0969.slice/crio-dbad34eb1fec68f7a7e336ad69b1f8b13d653856b96e361c02530b424fdd6c0f WatchSource:0}: Error finding container dbad34eb1fec68f7a7e336ad69b1f8b13d653856b96e361c02530b424fdd6c0f: Status 404 returned error can't find the container with id dbad34eb1fec68f7a7e336ad69b1f8b13d653856b96e361c02530b424fdd6c0f
Sep 11 11:53:13.878210 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:13.878189    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6f486dd7-7w98w"]
Sep 11 11:53:13.880446 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:13.880417    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7e598f_7a5e_4fdc_8d45_68f6d3f7d756.slice/crio-c8a9e9ddf8f3a05c91250d34d2ce60642f2c8d5b8f1a1f4b07dd45c64b4a6778 WatchSource:0}: Error finding container c8a9e9ddf8f3a05c91250d34d2ce60642f2c8d5b8f1a1f4b07dd45c64b4a6778: Status 404 returned error can't find the container with id c8a9e9ddf8f3a05c91250d34d2ce60642f2c8d5b8f1a1f4b07dd45c64b4a6778
Sep 11 11:53:13.883759 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:13.883732    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bd449b7f7-hbhw6"]
Sep 11 11:53:13.888936 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:13.888909    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:53:13.906992 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:13.906893    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbdbf3f_abe4_4f0c_830f_a47bb729254a.slice/crio-51ee741381ab01dfb6c16e4c8d61df4f0656f1e4deeef3b0c91e2094a2e85af3 WatchSource:0}: Error finding container 51ee741381ab01dfb6c16e4c8d61df4f0656f1e4deeef3b0c91e2094a2e85af3: Status 404 returned error can't find the container with id 51ee741381ab01dfb6c16e4c8d61df4f0656f1e4deeef3b0c91e2094a2e85af3
Sep 11 11:53:13.907388 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:13.907361    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e78e29_0d59_4a4c_ad37_11df77d4754a.slice/crio-f3f09a4a235701ae6b49887226987a2a15bb977433928b1e0ada6acf2c8bc985 WatchSource:0}: Error finding container f3f09a4a235701ae6b49887226987a2a15bb977433928b1e0ada6acf2c8bc985: Status 404 returned error can't find the container with id f3f09a4a235701ae6b49887226987a2a15bb977433928b1e0ada6acf2c8bc985
Sep 11 11:53:14.227668 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.227124    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-6dwvk" event={"ID":"fb40254a-26c2-495e-bc2c-48f1d22ed985","Type":"ContainerStarted","Data":"955e2079a4037a8e71276511ea3e0296cb3077b05340be75f74be7fe0683dd6a"}
Sep 11 11:53:14.227668 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.227623    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-6dwvk"
Sep 11 11:53:14.229714 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.229682    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerStarted","Data":"f3f09a4a235701ae6b49887226987a2a15bb977433928b1e0ada6acf2c8bc985"}
Sep 11 11:53:14.231289 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.231263    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs" event={"ID":"ac6fe090-abfb-42d6-995b-0ab2da0c0969","Type":"ContainerStarted","Data":"dbad34eb1fec68f7a7e336ad69b1f8b13d653856b96e361c02530b424fdd6c0f"}
Sep 11 11:53:14.233823 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.233792    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pxlwr" event={"ID":"22381901-1722-4032-888a-6ac1a3868149","Type":"ContainerStarted","Data":"055edc599951ab3dd7e9d3b731eeb729cfcfa4f1285913d0a32f90198ccc3fb6"}
Sep 11 11:53:14.234386 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.234362    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-6dwvk"
Sep 11 11:53:14.236996 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.236925    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrzn" event={"ID":"71251815-9411-41cb-bb13-30c51f158ebe","Type":"ContainerStarted","Data":"190c8e7f32408d8cfe9e391143433cd3db7cc0e8f1df286ff9a973e7ef316e9e"}
Sep 11 11:53:14.240345 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.240300    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd449b7f7-hbhw6" event={"ID":"9bbdbf3f-abe4-4f0c-830f-a47bb729254a","Type":"ContainerStarted","Data":"51ee741381ab01dfb6c16e4c8d61df4f0656f1e4deeef3b0c91e2094a2e85af3"}
Sep 11 11:53:14.242348 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.242299    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6f486dd7-7w98w" event={"ID":"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756","Type":"ContainerStarted","Data":"c8a9e9ddf8f3a05c91250d34d2ce60642f2c8d5b8f1a1f4b07dd45c64b4a6778"}
Sep 11 11:53:14.247297 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.247256    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-6dwvk" podStartSLOduration=2.031823719 podStartE2EDuration="19.2472431s" totalImagesPullingTime="17.215419381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:14.246099605 +0000 UTC m=+87.956770955" watchObservedRunningTime="2026-09-11 11:53:14.2472431 +0000 UTC m=+87.957914440"
Sep 11 11:53:14.420996 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.420958    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:14.519825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.519745    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:14.582008 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.581974    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:14.682790 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:14.682713    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:15.249020 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.248952    2586 generic.go:356] "Generic (PLEG): container finished" podID="71251815-9411-41cb-bb13-30c51f158ebe" containerID="7f623d862550f3c57614f8a5bef89e5692b346753bb14958499f3eb330140e7f" exitCode=0
Sep 11 11:53:15.251059 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.250850    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrzn" event={"ID":"71251815-9411-41cb-bb13-30c51f158ebe","Type":"ContainerDied","Data":"7f623d862550f3c57614f8a5bef89e5692b346753bb14958499f3eb330140e7f"}
Sep 11 11:53:15.798178 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.797471    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7c6789cfb-g5mtr"]
Sep 11 11:53:15.825921 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.825865    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c6789cfb-g5mtr"]
Sep 11 11:53:15.826070 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.825981    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:15.829222 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.829194    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 11:53:15.831004 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.830955    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 11:53:15.831216 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.831199    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6jdehb7814ghc\""
Sep 11 11:53:15.831536 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.831473    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zkbrf\""
Sep 11 11:53:15.839690 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.839371    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 11:53:15.842864 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.842663    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 11:53:15.933328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.932807    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-secret-metrics-server-client-certs\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:15.933328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.932876    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d5919dee-c964-496b-b29e-cbe63ba6fdfd-audit-log\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:15.933328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.932902    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l87qv\" (UniqueName: \"kubernetes.io/projected/d5919dee-c964-496b-b29e-cbe63ba6fdfd-kube-api-access-l87qv\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:15.933328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.932968    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d5919dee-c964-496b-b29e-cbe63ba6fdfd-metrics-server-audit-profiles\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:15.933328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.933018    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-client-ca-bundle\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:15.933328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.933072    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-secret-metrics-server-tls\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:15.933328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:15.933099    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5919dee-c964-496b-b29e-cbe63ba6fdfd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034157 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034117    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-secret-metrics-server-client-certs\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034322 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034176    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d5919dee-c964-496b-b29e-cbe63ba6fdfd-audit-log\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034322 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034198    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l87qv\" (UniqueName: \"kubernetes.io/projected/d5919dee-c964-496b-b29e-cbe63ba6fdfd-kube-api-access-l87qv\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034322 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034259    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d5919dee-c964-496b-b29e-cbe63ba6fdfd-metrics-server-audit-profiles\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034322 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034304    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-client-ca-bundle\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034576 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034343    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-secret-metrics-server-tls\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034576 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034373    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5919dee-c964-496b-b29e-cbe63ba6fdfd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.034903 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.034875    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d5919dee-c964-496b-b29e-cbe63ba6fdfd-audit-log\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.035058 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.035035    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5919dee-c964-496b-b29e-cbe63ba6fdfd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.035469 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.035428    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d5919dee-c964-496b-b29e-cbe63ba6fdfd-metrics-server-audit-profiles\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.038476 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.038360    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-client-ca-bundle\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.040514 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.040444    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-secret-metrics-server-tls\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.040750 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.040464    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d5919dee-c964-496b-b29e-cbe63ba6fdfd-secret-metrics-server-client-certs\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.043631 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.043605    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l87qv\" (UniqueName: \"kubernetes.io/projected/d5919dee-c964-496b-b29e-cbe63ba6fdfd-kube-api-access-l87qv\") pod \"metrics-server-7c6789cfb-g5mtr\" (UID: \"d5919dee-c964-496b-b29e-cbe63ba6fdfd\") " pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.144219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.144125    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:16.306838 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.306793    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"]
Sep 11 11:53:16.329530 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.329501    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"]
Sep 11 11:53:16.329698 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.329618    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.334046 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.333074    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tkd8r\""
Sep 11 11:53:16.334046 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.333312    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 11:53:16.423593 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.423518    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:16.441282 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.441249    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c293a891-acda-430a-b509-a0c05d9ac9e8-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-dbrcm\" (UID: \"c293a891-acda-430a-b509-a0c05d9ac9e8\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.441436 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.441293    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv52t\" (UniqueName: \"kubernetes.io/projected/c293a891-acda-430a-b509-a0c05d9ac9e8-kube-api-access-wv52t\") pod \"monitoring-plugin-5d85778b9d-dbrcm\" (UID: \"c293a891-acda-430a-b509-a0c05d9ac9e8\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.519897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.519758    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:16.543284 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.542780    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c293a891-acda-430a-b509-a0c05d9ac9e8-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-dbrcm\" (UID: \"c293a891-acda-430a-b509-a0c05d9ac9e8\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.544124 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.544046    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv52t\" (UniqueName: \"kubernetes.io/projected/c293a891-acda-430a-b509-a0c05d9ac9e8-kube-api-access-wv52t\") pod \"monitoring-plugin-5d85778b9d-dbrcm\" (UID: \"c293a891-acda-430a-b509-a0c05d9ac9e8\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.556842 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.556794    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv52t\" (UniqueName: \"kubernetes.io/projected/c293a891-acda-430a-b509-a0c05d9ac9e8-kube-api-access-wv52t\") pod \"monitoring-plugin-5d85778b9d-dbrcm\" (UID: \"c293a891-acda-430a-b509-a0c05d9ac9e8\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.563246 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.563216    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c293a891-acda-430a-b509-a0c05d9ac9e8-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-dbrcm\" (UID: \"c293a891-acda-430a-b509-a0c05d9ac9e8\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.622559 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.622425    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:16.646334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.646312    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:16.713373 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:16.713306    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bd449b7f7-hbhw6"]
Sep 11 11:53:17.591507 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.591455    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:17.698844 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.698737    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:17.743610 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.742925    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"]
Sep 11 11:53:17.748049 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.747574    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c6789cfb-g5mtr"]
Sep 11 11:53:17.748315 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.748292    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:53:17.779800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.779771    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:17.787336 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.787312    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:53:17.787454 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.787442    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.791334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.790942    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9zh5k\""
Sep 11 11:53:17.791334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.790988    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 11:53:17.791334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.790942    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 11:53:17.791334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.791211    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 11:53:17.792190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.792084    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 11:53:17.792190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.792110    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 11:53:17.792330 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.792296    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 11:53:17.793753 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.792619    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 11:53:17.793753 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.792994    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 11:53:17.793753 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.793547    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 11:53:17.793753 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.793673    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 11:53:17.794321 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.794165    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 11:53:17.796454 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.794700    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 11:53:17.796454 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.795688    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7a433t7ffc7q7\""
Sep 11 11:53:17.797055 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.796968    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 11:53:17.799171 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.799154    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 11:53:17.859881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.859611    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.859881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.859652    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.859881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.859695    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.859881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.859719    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.859881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.859755    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.859881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.859781    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860452    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860477    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860520    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860550    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860594    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6ch8\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-kube-api-access-p6ch8\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860614    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860634    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860653    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860672    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860692    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860713    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860727    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860743    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.860857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.860758    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.898718 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.898625    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962097    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6ch8\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-kube-api-access-p6ch8\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962166    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962199    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962227    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962259    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962291    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962324    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962350    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962376    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962402    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962457    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962496    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962541    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962565    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962604    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962631    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.963839 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962657    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.964773 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962686    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.964773 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962710    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.964773 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.962743    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.964773 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.963169    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.964987 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.964925    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.965446 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.965419    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.966816 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.966792    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.969266 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.967300    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.970278 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.969646    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.970278 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.970239    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.974939 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.974911    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.975412 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.975387    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.975517 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.975425    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.982285 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.982240    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.982385 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.982366    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.982997 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.982952    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.983661 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.983528    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.983661 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.983625    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.985134 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.985089    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.985859 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.985818    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.986633 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.986594    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.987037 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.986996    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.990186 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.990144    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6ch8\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-kube-api-access-p6ch8\") pod \"prometheus-k8s-0\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:17.995685 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:17.995629    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:18.108262 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.108228    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:18.275189 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.274223    2586 generic.go:356] "Generic (PLEG): container finished" podID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerID="9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269" exitCode=0
Sep 11 11:53:18.275189 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.274648    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269"}
Sep 11 11:53:18.277037 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.276763    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr" event={"ID":"d5919dee-c964-496b-b29e-cbe63ba6fdfd","Type":"ContainerStarted","Data":"93475394d93f2ea9850b66fd8f57375d81880e820e76cf1ed79ac4ad5633e463"}
Sep 11 11:53:18.280152 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.280048    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pxlwr" event={"ID":"22381901-1722-4032-888a-6ac1a3868149","Type":"ContainerStarted","Data":"d7bae5e2af4fcfa4132bfedf7f9fe1c4a1f24d9f4ce7ca2f0e2ce9c7516ed15e"}
Sep 11 11:53:18.283072 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.283047    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrzn" event={"ID":"71251815-9411-41cb-bb13-30c51f158ebe","Type":"ContainerStarted","Data":"d0c8f103d4947840aeb424b1b73ef7120dd33db454ecf22f1d9524c59eb2e6c6"}
Sep 11 11:53:18.283185 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.283096    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrzn" event={"ID":"71251815-9411-41cb-bb13-30c51f158ebe","Type":"ContainerStarted","Data":"368fe467ff8d80bb3541af24505c02ff7fdaf128627cf206ef66845da9baaf7c"}
Sep 11 11:53:18.285084 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.285036    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm" event={"ID":"c293a891-acda-430a-b509-a0c05d9ac9e8","Type":"ContainerStarted","Data":"b576efa15f41942669d55d588bcbe1e96231242190537176d1aff602d291df6e"}
Sep 11 11:53:18.319351 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.319300    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kxrzn" podStartSLOduration=6.006013003 podStartE2EDuration="7.319280926s" totalImagesPullingTime="1.20637s" totalInitContainerRuntime="106.897923ms" isStatefulPod=true podCreationTimestamp="2026-09-11 11:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:18.318523313 +0000 UTC m=+92.029194651" watchObservedRunningTime="2026-09-11 11:53:18.319280926 +0000 UTC m=+92.029952263"
Sep 11 11:53:18.337878 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:18.337828    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pxlwr" podStartSLOduration=33.953286947 podStartE2EDuration="45.337811793s" totalImagesPullingTime="11.384524846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:52:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:18.335203826 +0000 UTC m=+92.045875166" watchObservedRunningTime="2026-09-11 11:53:18.337811793 +0000 UTC m=+92.048483131"
Sep 11 11:53:20.304218 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:20.304166    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:20.415092 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:20.415003    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:20.469295 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:20.469239    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:20.628738 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:20.627682    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:53:20.630344 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:53:20.630312    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f24b0d4_e719_40c2_9422_5c71b3d7b1a3.slice/crio-522277095e5515a03326092f00a0634cda0c7b5d831f5746c0989ccc08546e5b WatchSource:0}: Error finding container 522277095e5515a03326092f00a0634cda0c7b5d831f5746c0989ccc08546e5b: Status 404 returned error can't find the container with id 522277095e5515a03326092f00a0634cda0c7b5d831f5746c0989ccc08546e5b
Sep 11 11:53:21.302561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:21.302526    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerStarted","Data":"522277095e5515a03326092f00a0634cda0c7b5d831f5746c0989ccc08546e5b"}
Sep 11 11:53:21.305212 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:21.305176    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs" event={"ID":"ac6fe090-abfb-42d6-995b-0ab2da0c0969","Type":"ContainerStarted","Data":"b75a5978a3b850051f6e67c7cbde7879d32d9fa58213fb6878180f06d6b0de4c"}
Sep 11 11:53:21.307903 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:21.307346    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd449b7f7-hbhw6" event={"ID":"9bbdbf3f-abe4-4f0c-830f-a47bb729254a","Type":"ContainerStarted","Data":"e9ac524d4a342f8dedfe1582ff5f39b7c58937e862917237212b055c443557f6"}
Sep 11 11:53:21.310201 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:21.310172    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6f486dd7-7w98w" event={"ID":"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756","Type":"ContainerStarted","Data":"a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7"}
Sep 11 11:53:21.327781 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:21.326621    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bd449b7f7-hbhw6" podStartSLOduration=8.707803052 podStartE2EDuration="15.326602869s" totalImagesPullingTime="6.618799817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:21.326099322 +0000 UTC m=+95.036770661" watchObservedRunningTime="2026-09-11 11:53:21.326602869 +0000 UTC m=+95.037274208"
Sep 11 11:53:21.345073 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:21.345026    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6f486dd7-7w98w" podStartSLOduration=6.912514865 podStartE2EDuration="13.345013783s" totalImagesPullingTime="6.432498918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:21.343847914 +0000 UTC m=+95.054519255" watchObservedRunningTime="2026-09-11 11:53:21.345013783 +0000 UTC m=+95.055685120"
Sep 11 11:53:22.697732 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:22.697707    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6f486dd7-7w98w"]
Sep 11 11:53:23.321434 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.320990    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr" event={"ID":"d5919dee-c964-496b-b29e-cbe63ba6fdfd","Type":"ContainerStarted","Data":"6f04429f4439bd98af849121a31299af2b374c0fa22fa8125ede685b3872c224"}
Sep 11 11:53:23.322855 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.322832    2586 generic.go:356] "Generic (PLEG): container finished" podID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16" exitCode=0
Sep 11 11:53:23.322950 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.322924    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"}
Sep 11 11:53:23.324749 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.324717    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm" event={"ID":"c293a891-acda-430a-b509-a0c05d9ac9e8","Type":"ContainerStarted","Data":"c820a8bfcf71a40515dae372ec1bd31baac36ae1c0aa119fd9cd5d53b8da647d"}
Sep 11 11:53:23.325379 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.325363    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:23.328780 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.328754    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerStarted","Data":"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"}
Sep 11 11:53:23.328891 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.328820    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerStarted","Data":"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"}
Sep 11 11:53:23.328891 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.328861    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerStarted","Data":"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"}
Sep 11 11:53:23.331168 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.331145    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm"
Sep 11 11:53:23.331712 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.331691    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs" event={"ID":"ac6fe090-abfb-42d6-995b-0ab2da0c0969","Type":"ContainerStarted","Data":"9597d95256fe918ad73381015221bc1597c4b3f2e37640fe01905e2f182650f2"}
Sep 11 11:53:23.331792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.331734    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs" event={"ID":"ac6fe090-abfb-42d6-995b-0ab2da0c0969","Type":"ContainerStarted","Data":"55800a6fbd320da164b692b695d4155713952a0cb8e4ccf8850838f7fda282b1"}
Sep 11 11:53:23.340175 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.340136    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr" podStartSLOduration=3.176672386 podStartE2EDuration="8.340119706s" totalImagesPullingTime="5.16344732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:23.339466061 +0000 UTC m=+97.050137401" watchObservedRunningTime="2026-09-11 11:53:23.340119706 +0000 UTC m=+97.050791045"
Sep 11 11:53:23.354577 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.354530    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dbrcm" podStartSLOduration=2.18164935 podStartE2EDuration="7.35451675s" totalImagesPullingTime="5.1728674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:23.353464188 +0000 UTC m=+97.064135527" watchObservedRunningTime="2026-09-11 11:53:23.35451675 +0000 UTC m=+97.065188085"
Sep 11 11:53:23.415117 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:23.415065    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-zcdrs" podStartSLOduration=5.950167028 podStartE2EDuration="12.415042502s" totalImagesPullingTime="6.464875474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:23.41263461 +0000 UTC m=+97.123305951" watchObservedRunningTime="2026-09-11 11:53:23.415042502 +0000 UTC m=+97.125713842"
Sep 11 11:53:24.126636 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:24.126578    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fndk9"
Sep 11 11:53:24.341561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:24.341524    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerStarted","Data":"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"}
Sep 11 11:53:24.341739 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:24.341619    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerStarted","Data":"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"}
Sep 11 11:53:25.350586 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:25.350547    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerStarted","Data":"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"}
Sep 11 11:53:25.379588 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:25.379542    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.923553272 podStartE2EDuration="13.379526037s" totalImagesPullingTime="9.421822375s" totalInitContainerRuntime="34.15039ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:25.378985609 +0000 UTC m=+99.089656948" watchObservedRunningTime="2026-09-11 11:53:25.379526037 +0000 UTC m=+99.090197377"
Sep 11 11:53:26.355832 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:26.355796    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:26.406618 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:26.406578    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:26.816319 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:26.816295    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:27.365351 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:27.365266    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerStarted","Data":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"}
Sep 11 11:53:27.365736 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:27.365371    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerStarted","Data":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"}
Sep 11 11:53:27.710282 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:27.710204    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:27.764658 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:27.764626    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:28.606029 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:28.605996    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:29.837504 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:29.837450    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:29.887895 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:29.887873    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:30.379995 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:30.379972    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerStarted","Data":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"}
Sep 11 11:53:30.380051 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:30.380033    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerStarted","Data":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"}
Sep 11 11:53:31.392698 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:31.392664    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerStarted","Data":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"}
Sep 11 11:53:31.393154 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:31.392754    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerStarted","Data":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"}
Sep 11 11:53:31.423720 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:31.423667    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.863288347 podStartE2EDuration="14.423648982s" totalImagesPullingTime="6.537570648s" totalInitContainerRuntime="22.789987ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:31.420185811 +0000 UTC m=+105.130857188" watchObservedRunningTime="2026-09-11 11:53:31.423648982 +0000 UTC m=+105.134320342"
Sep 11 11:53:31.477206 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:31.477176    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:31.577541 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:31.577518    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:31.677976 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:31.677923    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:33.105533 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:33.105504    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:33.109233 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:33.109216    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:53:33.166390 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:33.166367    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:33.496505 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:33.496420    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:33.720909 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:33.720868    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:33.920550 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:33.920519    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:34.020976 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:34.020945    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:35.366981 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:35.366948    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:35.467540 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:35.467515    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:36.145041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:36.145006    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:36.145212 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:36.145070    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:39.488396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:39.488357    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:39.538507 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:39.538459    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:43.782874 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:43.782834    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:43.883802 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:43.883767    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:43.938552 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:43.938521    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:46.334264 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.334214    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bd449b7f7-hbhw6" podUID="9bbdbf3f-abe4-4f0c-830f-a47bb729254a" containerName="console" containerID="cri-o://e9ac524d4a342f8dedfe1582ff5f39b7c58937e862917237212b055c443557f6" gracePeriod=15
Sep 11 11:53:46.459627 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.459596    2586 generic.go:356] "Generic (PLEG): container finished" podID="9bbdbf3f-abe4-4f0c-830f-a47bb729254a" containerID="e9ac524d4a342f8dedfe1582ff5f39b7c58937e862917237212b055c443557f6" exitCode=0
Sep 11 11:53:46.459743 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.459667    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd449b7f7-hbhw6" event={"ID":"9bbdbf3f-abe4-4f0c-830f-a47bb729254a","Type":"ContainerDied","Data":"e9ac524d4a342f8dedfe1582ff5f39b7c58937e862917237212b055c443557f6"}
Sep 11 11:53:46.573943 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.573921    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:46.591997 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.591939    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-service-ca\") pod \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") "
Sep 11 11:53:46.591997 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.591988    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-serving-cert\") pod \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") "
Sep 11 11:53:46.592166 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.592043    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-oauth-serving-cert\") pod \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") "
Sep 11 11:53:46.592166 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.592080    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-config\") pod \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") "
Sep 11 11:53:46.592166 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.592115    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-oauth-config\") pod \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") "
Sep 11 11:53:46.592166 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.592144    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxtvz\" (UniqueName: \"kubernetes.io/projected/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-kube-api-access-wxtvz\") pod \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\" (UID: \"9bbdbf3f-abe4-4f0c-830f-a47bb729254a\") "
Sep 11 11:53:46.592477 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.592445    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9bbdbf3f-abe4-4f0c-830f-a47bb729254a" (UID: "9bbdbf3f-abe4-4f0c-830f-a47bb729254a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:46.592595 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.592467    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-config" (OuterVolumeSpecName: "console-config") pod "9bbdbf3f-abe4-4f0c-830f-a47bb729254a" (UID: "9bbdbf3f-abe4-4f0c-830f-a47bb729254a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:46.592765 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.592740    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-service-ca" (OuterVolumeSpecName: "service-ca") pod "9bbdbf3f-abe4-4f0c-830f-a47bb729254a" (UID: "9bbdbf3f-abe4-4f0c-830f-a47bb729254a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:46.594763 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.594720    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9bbdbf3f-abe4-4f0c-830f-a47bb729254a" (UID: "9bbdbf3f-abe4-4f0c-830f-a47bb729254a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:53:46.595389 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.595364    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9bbdbf3f-abe4-4f0c-830f-a47bb729254a" (UID: "9bbdbf3f-abe4-4f0c-830f-a47bb729254a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:53:46.595476 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.595457    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-kube-api-access-wxtvz" (OuterVolumeSpecName: "kube-api-access-wxtvz") pod "9bbdbf3f-abe4-4f0c-830f-a47bb729254a" (UID: "9bbdbf3f-abe4-4f0c-830f-a47bb729254a"). InnerVolumeSpecName "kube-api-access-wxtvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:53:46.693286 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.693262    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:46.693286 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.693284    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-oauth-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:46.693429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.693298    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxtvz\" (UniqueName: \"kubernetes.io/projected/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-kube-api-access-wxtvz\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:46.693429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.693314    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:46.693429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.693328    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-console-serving-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:46.693429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:46.693340    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bbdbf3f-abe4-4f0c-830f-a47bb729254a-oauth-serving-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:47.467803 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:47.467756    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd449b7f7-hbhw6" event={"ID":"9bbdbf3f-abe4-4f0c-830f-a47bb729254a","Type":"ContainerDied","Data":"51ee741381ab01dfb6c16e4c8d61df4f0656f1e4deeef3b0c91e2094a2e85af3"}
Sep 11 11:53:47.468259 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:47.467863    2586 scope.go:117] "RemoveContainer" containerID="e9ac524d4a342f8dedfe1582ff5f39b7c58937e862917237212b055c443557f6"
Sep 11 11:53:47.468259 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:47.468032    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd449b7f7-hbhw6"
Sep 11 11:53:47.485824 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:47.485782    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bd449b7f7-hbhw6"]
Sep 11 11:53:47.487171 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:47.487147    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bd449b7f7-hbhw6"]
Sep 11 11:53:48.365432 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.365368    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c6f486dd7-7w98w" podUID="fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" containerName="console" containerID="cri-o://a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7" gracePeriod=15
Sep 11 11:53:48.489128 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.489103    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:48.539759 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.539736    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:48.595268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.595248    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:48.610474 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610453    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-service-ca\") pod \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") "
Sep 11 11:53:48.610582 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610478    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-config\") pod \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") "
Sep 11 11:53:48.610582 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610524    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-serving-cert\") pod \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") "
Sep 11 11:53:48.610582 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610551    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-trusted-ca-bundle\") pod \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") "
Sep 11 11:53:48.610582 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610569    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-oauth-serving-cert\") pod \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") "
Sep 11 11:53:48.610792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610611    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-oauth-config\") pod \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") "
Sep 11 11:53:48.610792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610659    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw9fw\" (UniqueName: \"kubernetes.io/projected/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-kube-api-access-qw9fw\") pod \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\" (UID: \"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756\") "
Sep 11 11:53:48.610971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610942    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-service-ca" (OuterVolumeSpecName: "service-ca") pod "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" (UID: "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:48.610971 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610958    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-config" (OuterVolumeSpecName: "console-config") pod "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" (UID: "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:48.611109 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.610970    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" (UID: "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:48.611232 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.611213    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" (UID: "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:48.613205 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.613176    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" (UID: "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:53:48.613307 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.613245    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" (UID: "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:53:48.613453 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.613422    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-kube-api-access-qw9fw" (OuterVolumeSpecName: "kube-api-access-qw9fw") pod "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" (UID: "fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756"). InnerVolumeSpecName "kube-api-access-qw9fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:53:48.711960 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.711931    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:48.711960 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.711953    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:48.711960 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.711963    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-serving-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:48.712122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.711972    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-trusted-ca-bundle\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:48.712122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.711981    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-oauth-serving-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:48.712122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.711989    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-console-oauth-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:48.712122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.711997    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw9fw\" (UniqueName: \"kubernetes.io/projected/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756-kube-api-access-qw9fw\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:53:48.822236 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:48.822195    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bbdbf3f-abe4-4f0c-830f-a47bb729254a" path="/var/lib/kubelet/pods/9bbdbf3f-abe4-4f0c-830f-a47bb729254a/volumes"
Sep 11 11:53:49.478299 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.478262    2586 generic.go:356] "Generic (PLEG): container finished" podID="fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" containerID="a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7" exitCode=0
Sep 11 11:53:49.478396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.478309    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6f486dd7-7w98w"
Sep 11 11:53:49.478396 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.478356    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6f486dd7-7w98w" event={"ID":"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756","Type":"ContainerDied","Data":"a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7"}
Sep 11 11:53:49.478513 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.478407    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6f486dd7-7w98w" event={"ID":"fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756","Type":"ContainerDied","Data":"c8a9e9ddf8f3a05c91250d34d2ce60642f2c8d5b8f1a1f4b07dd45c64b4a6778"}
Sep 11 11:53:49.478513 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.478435    2586 scope.go:117] "RemoveContainer" containerID="a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7"
Sep 11 11:53:49.480542 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.480508    2586 generic.go:356] "Generic (PLEG): container finished" podID="1b97718f-6b7f-4b58-b64f-dc1ebed0ce34" containerID="6cdb71b3d605d0ba6c146b470f9165f761dff8e23b0733c98485267494ed477c" exitCode=0
Sep 11 11:53:49.480662 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.480560    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f" event={"ID":"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34","Type":"ContainerDied","Data":"6cdb71b3d605d0ba6c146b470f9165f761dff8e23b0733c98485267494ed477c"}
Sep 11 11:53:49.480909 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.480891    2586 scope.go:117] "RemoveContainer" containerID="6cdb71b3d605d0ba6c146b470f9165f761dff8e23b0733c98485267494ed477c"
Sep 11 11:53:49.487219 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.487195    2586 scope.go:117] "RemoveContainer" containerID="a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7"
Sep 11 11:53:49.487636 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:53:49.487610    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7\": container with ID starting with a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7 not found: ID does not exist" containerID="a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7"
Sep 11 11:53:49.487772 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.487647    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7"} err="failed to get container status \"a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7\": rpc error: code = NotFound desc = could not find container \"a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7\": container with ID starting with a4d958006578d8c02a2650512f7b9482de425ffd15f674f56cb7a74e691d5bd7 not found: ID does not exist"
Sep 11 11:53:49.516756 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.516714    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6f486dd7-7w98w"]
Sep 11 11:53:49.519918 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:49.519896    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c6f486dd7-7w98w"]
Sep 11 11:53:50.486795 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:50.486754    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gnh4f" event={"ID":"1b97718f-6b7f-4b58-b64f-dc1ebed0ce34","Type":"ContainerStarted","Data":"d33f9f1b59b54cbae6adf54fcedb1889cf5cf4e065e64f8941feae3ada21ce18"}
Sep 11 11:53:50.823039 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:50.823009    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" path="/var/lib/kubelet/pods/fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756/volumes"
Sep 11 11:53:56.152041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:56.151698    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:53:56.163712 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:53:56.163693    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7c6789cfb-g5mtr"
Sep 11 11:54:03.500833 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:03.500775    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:03.549737 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:03.549699    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:18.109277 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:18.109227    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:18.126409 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:18.126384    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:18.629678 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:18.629647    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:27.616874 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:27.616834    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:27.666049 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:27.666020    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:31.848709 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:31.848671    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:54:31.849191 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:31.848941    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="alertmanager" containerID="cri-o://8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6" gracePeriod=120
Sep 11 11:54:31.849191 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:31.849008    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-metric" containerID="cri-o://bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5" gracePeriod=120
Sep 11 11:54:31.849191 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:31.849041    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-web" containerID="cri-o://1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c" gracePeriod=120
Sep 11 11:54:31.849191 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:31.849086    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy" containerID="cri-o://11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95" gracePeriod=120
Sep 11 11:54:31.849191 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:31.849050    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="config-reloader" containerID="cri-o://1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231" gracePeriod=120
Sep 11 11:54:31.849191 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:31.849098    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="prom-label-proxy" containerID="cri-o://daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715" gracePeriod=120
Sep 11 11:54:32.676111 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676076    2586 generic.go:356] "Generic (PLEG): container finished" podID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerID="daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715" exitCode=0
Sep 11 11:54:32.676111 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676106    2586 generic.go:356] "Generic (PLEG): container finished" podID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerID="11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95" exitCode=0
Sep 11 11:54:32.676111 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676114    2586 generic.go:356] "Generic (PLEG): container finished" podID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerID="1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231" exitCode=0
Sep 11 11:54:32.676111 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676123    2586 generic.go:356] "Generic (PLEG): container finished" podID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerID="8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6" exitCode=0
Sep 11 11:54:32.676404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676166    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"}
Sep 11 11:54:32.676404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676252    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"}
Sep 11 11:54:32.676404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676282    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"}
Sep 11 11:54:32.676404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:32.676309    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"}
Sep 11 11:54:33.105790 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.105764    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.249822 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.249747    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.249822 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.249777    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.250138 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.250121    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-metrics-client-ca\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.250631 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.250310    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-web-config\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.250631 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.250345    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-volume\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.250631 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.250412    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-main-db\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.250631 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.250566    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:33.251516 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.251034    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:54:33.252894 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.252835    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253048    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nztw\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-kube-api-access-6nztw\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253087    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-tls-assets\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253140    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-cluster-tls-config\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253171    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-out\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253200    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-web\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253247    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-main-tls\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253277    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-trusted-ca-bundle\") pod \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\" (UID: \"a0e78e29-0d59-4a4c-ad37-11df77d4754a\") "
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253676    2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-main-db\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253698    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.253902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.253712    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-metrics-client-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.254611 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.254582    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:33.259269 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.258613    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-volume" (OuterVolumeSpecName: "config-volume") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:33.259269 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.259153    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:33.260087 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.260043    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:33.260173 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.260120    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-kube-api-access-6nztw" (OuterVolumeSpecName: "kube-api-access-6nztw") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "kube-api-access-6nztw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:54:33.260236 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.260170    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:33.260284 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.260047    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:54:33.260822 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.260797    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-out" (OuterVolumeSpecName: "config-out") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:54:33.264183 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.264163    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:33.268089 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.268071    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-web-config" (OuterVolumeSpecName: "web-config") pod "a0e78e29-0d59-4a4c-ad37-11df77d4754a" (UID: "a0e78e29-0d59-4a4c-ad37-11df77d4754a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:33.354532 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354507    2586 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-cluster-tls-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354532 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354529    2586 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-out\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354545    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354559    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-main-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354572    2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0e78e29-0d59-4a4c-ad37-11df77d4754a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354585    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354598    2586 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-web-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354610    2586 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a0e78e29-0d59-4a4c-ad37-11df77d4754a-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354622    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nztw\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-kube-api-access-6nztw\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.354671 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.354635    2586 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0e78e29-0d59-4a4c-ad37-11df77d4754a-tls-assets\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:33.683916 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.683892    2586 generic.go:356] "Generic (PLEG): container finished" podID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerID="bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5" exitCode=0
Sep 11 11:54:33.683916 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.683914    2586 generic.go:356] "Generic (PLEG): container finished" podID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerID="1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c" exitCode=0
Sep 11 11:54:33.684137 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.683961    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"}
Sep 11 11:54:33.684137 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.683984    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.684137 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.684077    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"}
Sep 11 11:54:33.684137 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.684129    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a0e78e29-0d59-4a4c-ad37-11df77d4754a","Type":"ContainerDied","Data":"f3f09a4a235701ae6b49887226987a2a15bb977433928b1e0ada6acf2c8bc985"}
Sep 11 11:54:33.684293 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.684146    2586 scope.go:117] "RemoveContainer" containerID="daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"
Sep 11 11:54:33.692065 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.692045    2586 scope.go:117] "RemoveContainer" containerID="bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"
Sep 11 11:54:33.699413 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.699399    2586 scope.go:117] "RemoveContainer" containerID="11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"
Sep 11 11:54:33.706350 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.706313    2586 scope.go:117] "RemoveContainer" containerID="1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"
Sep 11 11:54:33.708069 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.708052    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:54:33.712984 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.712965    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:54:33.714078 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.714049    2586 scope.go:117] "RemoveContainer" containerID="1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"
Sep 11 11:54:33.720953 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.720939    2586 scope.go:117] "RemoveContainer" containerID="8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"
Sep 11 11:54:33.727862 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.727847    2586 scope.go:117] "RemoveContainer" containerID="9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269"
Sep 11 11:54:33.735251 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.735235    2586 scope.go:117] "RemoveContainer" containerID="daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"
Sep 11 11:54:33.735534 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:33.735518    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715\": container with ID starting with daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715 not found: ID does not exist" containerID="daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"
Sep 11 11:54:33.735588 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.735541    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"} err="failed to get container status \"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715\": rpc error: code = NotFound desc = could not find container \"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715\": container with ID starting with daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715 not found: ID does not exist"
Sep 11 11:54:33.735588 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.735557    2586 scope.go:117] "RemoveContainer" containerID="bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"
Sep 11 11:54:33.735797 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:33.735779    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5\": container with ID starting with bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5 not found: ID does not exist" containerID="bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"
Sep 11 11:54:33.735858 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.735806    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"} err="failed to get container status \"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5\": rpc error: code = NotFound desc = could not find container \"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5\": container with ID starting with bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5 not found: ID does not exist"
Sep 11 11:54:33.735858 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.735829    2586 scope.go:117] "RemoveContainer" containerID="11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"
Sep 11 11:54:33.736084 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:33.736066    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95\": container with ID starting with 11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95 not found: ID does not exist" containerID="11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"
Sep 11 11:54:33.736123 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736090    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"} err="failed to get container status \"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95\": rpc error: code = NotFound desc = could not find container \"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95\": container with ID starting with 11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95 not found: ID does not exist"
Sep 11 11:54:33.736123 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736104    2586 scope.go:117] "RemoveContainer" containerID="1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"
Sep 11 11:54:33.736313 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:33.736297    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c\": container with ID starting with 1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c not found: ID does not exist" containerID="1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"
Sep 11 11:54:33.736389 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736316    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"} err="failed to get container status \"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c\": rpc error: code = NotFound desc = could not find container \"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c\": container with ID starting with 1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c not found: ID does not exist"
Sep 11 11:54:33.736389 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736330    2586 scope.go:117] "RemoveContainer" containerID="1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"
Sep 11 11:54:33.736610 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:33.736594    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231\": container with ID starting with 1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231 not found: ID does not exist" containerID="1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"
Sep 11 11:54:33.736678 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736616    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"} err="failed to get container status \"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231\": rpc error: code = NotFound desc = could not find container \"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231\": container with ID starting with 1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231 not found: ID does not exist"
Sep 11 11:54:33.736678 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736636    2586 scope.go:117] "RemoveContainer" containerID="8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"
Sep 11 11:54:33.736887 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:33.736871    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6\": container with ID starting with 8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6 not found: ID does not exist" containerID="8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"
Sep 11 11:54:33.736927 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736890    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"} err="failed to get container status \"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6\": rpc error: code = NotFound desc = could not find container \"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6\": container with ID starting with 8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6 not found: ID does not exist"
Sep 11 11:54:33.736927 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.736903    2586 scope.go:117] "RemoveContainer" containerID="9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269"
Sep 11 11:54:33.737130 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:33.737115    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269\": container with ID starting with 9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269 not found: ID does not exist" containerID="9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269"
Sep 11 11:54:33.737171 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737134    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269"} err="failed to get container status \"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269\": rpc error: code = NotFound desc = could not find container \"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269\": container with ID starting with 9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269 not found: ID does not exist"
Sep 11 11:54:33.737171 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737148    2586 scope.go:117] "RemoveContainer" containerID="daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"
Sep 11 11:54:33.737377 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737360    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715"} err="failed to get container status \"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715\": rpc error: code = NotFound desc = could not find container \"daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715\": container with ID starting with daa4c1201453236d3ca4c59922ff4bb901990fb9cbf2330640f5437e22735715 not found: ID does not exist"
Sep 11 11:54:33.737437 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737379    2586 scope.go:117] "RemoveContainer" containerID="bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"
Sep 11 11:54:33.737632 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737613    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5"} err="failed to get container status \"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5\": rpc error: code = NotFound desc = could not find container \"bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5\": container with ID starting with bd64ba81dd158ed4a5ee458b97346e7eef93971f91bfbf6c49ef3cc7e5f565d5 not found: ID does not exist"
Sep 11 11:54:33.737688 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737633    2586 scope.go:117] "RemoveContainer" containerID="11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"
Sep 11 11:54:33.737840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737824    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95"} err="failed to get container status \"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95\": rpc error: code = NotFound desc = could not find container \"11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95\": container with ID starting with 11d7ebb9798d0f8e1362d44f6f0d2da310a3f2df582aac191c36fcf6f82f8d95 not found: ID does not exist"
Sep 11 11:54:33.737908 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.737842    2586 scope.go:117] "RemoveContainer" containerID="1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"
Sep 11 11:54:33.738077 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.738059    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c"} err="failed to get container status \"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c\": rpc error: code = NotFound desc = could not find container \"1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c\": container with ID starting with 1fed34dfb75a9d899fdb5ed7675221ba458a60210813c6959a4496e40182297c not found: ID does not exist"
Sep 11 11:54:33.738122 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.738078    2586 scope.go:117] "RemoveContainer" containerID="1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"
Sep 11 11:54:33.738561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.738462    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231"} err="failed to get container status \"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231\": rpc error: code = NotFound desc = could not find container \"1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231\": container with ID starting with 1bf79dafda300d3246078821577d218d8274fbd35bba6f7fd32b0977dac7d231 not found: ID does not exist"
Sep 11 11:54:33.738561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.738509    2586 scope.go:117] "RemoveContainer" containerID="8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"
Sep 11 11:54:33.738968 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.738928    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6"} err="failed to get container status \"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6\": rpc error: code = NotFound desc = could not find container \"8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6\": container with ID starting with 8b7d6d96015163826c253ab0e05c10f81c02db61d451864478c852d9923a56a6 not found: ID does not exist"
Sep 11 11:54:33.738968 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.738953    2586 scope.go:117] "RemoveContainer" containerID="9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269"
Sep 11 11:54:33.739303 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.739282    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269"} err="failed to get container status \"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269\": rpc error: code = NotFound desc = could not find container \"9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269\": container with ID starting with 9ed7484f7d89e975a02235df905608ffee7993637c8fe48c1b7542e84370b269 not found: ID does not exist"
Sep 11 11:54:33.876347 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:33.876320    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:34.822841 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:34.822805    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" path="/var/lib/kubelet/pods/a0e78e29-0d59-4a4c-ad37-11df77d4754a/volumes"
Sep 11 11:54:36.087255 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.087222    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:54:36.087703 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.087589    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="prometheus" containerID="cri-o://6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85" gracePeriod=600
Sep 11 11:54:36.087703 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.087623    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy" containerID="cri-o://67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d" gracePeriod=600
Sep 11 11:54:36.087703 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.087595    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="thanos-sidecar" containerID="cri-o://5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b" gracePeriod=600
Sep 11 11:54:36.087840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.087626    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="config-reloader" containerID="cri-o://4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82" gracePeriod=600
Sep 11 11:54:36.087840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.087642    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4" gracePeriod=600
Sep 11 11:54:36.087840 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.087635    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-web" containerID="cri-o://b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504" gracePeriod=600
Sep 11 11:54:36.320828 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.320806    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.478577 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478477    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-trusted-ca-bundle\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478577 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478544    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-metrics-client-certs\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478578    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478610    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-grpc-tls\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478638    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-serving-certs-ca-bundle\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478680    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-kubelet-serving-ca-bundle\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478729    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-db\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478753    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6ch8\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-kube-api-access-p6ch8\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.478792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478783    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-kube-rbac-proxy\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478813    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config-out\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478838    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-web-config\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478868    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-0\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478894    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-metrics-client-ca\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478958    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478980    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-tls-assets\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.478992    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479003    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-tls\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479055    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-2\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479065    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479081    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479110    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-1\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.479136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479139    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-thanos-prometheus-http-client-file\") pod \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\" (UID: \"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3\") "
Sep 11 11:54:36.480061 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479393    2586 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.480061 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479414    2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.481658 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.479070    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:36.481658 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.480387    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:54:36.481658 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.480981    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:36.481884 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.481097    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:36.481884 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.481476    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:36.481884 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.481587    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.481884 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.481607    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.481884 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.481630    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.482113 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.481995    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.482165 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.482124    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:54:36.482311 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.482278    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.483051 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.483025    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.483524 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.483501    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config-out" (OuterVolumeSpecName: "config-out") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:54:36.483705 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.483688    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.483830 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.483808    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:54:36.484024 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.484009    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config" (OuterVolumeSpecName: "config") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.484281 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.484263    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-kube-api-access-p6ch8" (OuterVolumeSpecName: "kube-api-access-p6ch8") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "kube-api-access-p6ch8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:54:36.493632 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.493607    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-web-config" (OuterVolumeSpecName: "web-config") pod "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" (UID: "5f24b0d4-e719-40c2-9422-5c71b3d7b1a3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:54:36.580014 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.579992    2586 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-kube-rbac-proxy\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580014 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580013    2586 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config-out\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580022    2586 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-web-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580030    2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580038    2586 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-metrics-client-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580047    2586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580057    2586 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-tls-assets\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580065    2586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580075    2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580084    2586 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580092    2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580100    2586 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580109    2586 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-metrics-client-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580117    2586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580126    2586 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-secret-grpc-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580477 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580134    2586 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580477 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580142    2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-prometheus-k8s-db\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.580477 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.580150    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6ch8\" (UniqueName: \"kubernetes.io/projected/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3-kube-api-access-p6ch8\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:54:36.701377 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701347    2586 generic.go:356] "Generic (PLEG): container finished" podID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4" exitCode=0
Sep 11 11:54:36.701377 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701374    2586 generic.go:356] "Generic (PLEG): container finished" podID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d" exitCode=0
Sep 11 11:54:36.701522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701384    2586 generic.go:356] "Generic (PLEG): container finished" podID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504" exitCode=0
Sep 11 11:54:36.701522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701392    2586 generic.go:356] "Generic (PLEG): container finished" podID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b" exitCode=0
Sep 11 11:54:36.701522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701402    2586 generic.go:356] "Generic (PLEG): container finished" podID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82" exitCode=0
Sep 11 11:54:36.701522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701411    2586 generic.go:356] "Generic (PLEG): container finished" podID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85" exitCode=0
Sep 11 11:54:36.701522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701412    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"}
Sep 11 11:54:36.701522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701452    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.701522 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701506    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"}
Sep 11 11:54:36.701825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701539    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"}
Sep 11 11:54:36.701825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701577    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"}
Sep 11 11:54:36.701825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701588    2586 scope.go:117] "RemoveContainer" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.701825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701605    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"}
Sep 11 11:54:36.701825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701633    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"}
Sep 11 11:54:36.701825 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.701659    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f24b0d4-e719-40c2-9422-5c71b3d7b1a3","Type":"ContainerDied","Data":"522277095e5515a03326092f00a0634cda0c7b5d831f5746c0989ccc08546e5b"}
Sep 11 11:54:36.710571 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.710541    2586 scope.go:117] "RemoveContainer" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.718662 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.718644    2586 scope.go:117] "RemoveContainer" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.725436 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.725415    2586 scope.go:117] "RemoveContainer" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.730411 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.730353    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:54:36.734040 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.734017    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:54:36.734271 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.734249    2586 scope.go:117] "RemoveContainer" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.741046 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.741027    2586 scope.go:117] "RemoveContainer" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.748395 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.748362    2586 scope.go:117] "RemoveContainer" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.755169 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.755154    2586 scope.go:117] "RemoveContainer" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.755438 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:36.755419    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": container with ID starting with 8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4 not found: ID does not exist" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.755509 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.755444    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"} err="failed to get container status \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": rpc error: code = NotFound desc = could not find container \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": container with ID starting with 8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4 not found: ID does not exist"
Sep 11 11:54:36.755509 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.755461    2586 scope.go:117] "RemoveContainer" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.755796 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:36.755768    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": container with ID starting with 67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d not found: ID does not exist" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.755796 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.755805    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"} err="failed to get container status \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": rpc error: code = NotFound desc = could not find container \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": container with ID starting with 67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d not found: ID does not exist"
Sep 11 11:54:36.756033 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.755829    2586 scope.go:117] "RemoveContainer" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.756287 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:36.756260    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": container with ID starting with b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504 not found: ID does not exist" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.756398 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.756293    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"} err="failed to get container status \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": rpc error: code = NotFound desc = could not find container \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": container with ID starting with b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504 not found: ID does not exist"
Sep 11 11:54:36.756398 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.756312    2586 scope.go:117] "RemoveContainer" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.756706 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:36.756669    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": container with ID starting with 5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b not found: ID does not exist" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.756778 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.756709    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"} err="failed to get container status \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": rpc error: code = NotFound desc = could not find container \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": container with ID starting with 5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b not found: ID does not exist"
Sep 11 11:54:36.756778 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.756732    2586 scope.go:117] "RemoveContainer" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.757085 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:36.757056    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": container with ID starting with 4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82 not found: ID does not exist" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.757147 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.757089    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"} err="failed to get container status \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": rpc error: code = NotFound desc = could not find container \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": container with ID starting with 4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82 not found: ID does not exist"
Sep 11 11:54:36.757285 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.757257    2586 scope.go:117] "RemoveContainer" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.757864 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:36.757836    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": container with ID starting with 6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85 not found: ID does not exist" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.757962 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.757880    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"} err="failed to get container status \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": rpc error: code = NotFound desc = could not find container \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": container with ID starting with 6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85 not found: ID does not exist"
Sep 11 11:54:36.757962 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.757903    2586 scope.go:117] "RemoveContainer" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759152    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759744    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759762    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759785    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759794    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759807    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbdbf3f-abe4-4f0c-830f-a47bb729254a" containerName="console"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759823    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbdbf3f-abe4-4f0c-830f-a47bb729254a" containerName="console"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759838    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759848    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759865    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-web"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759874    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-web"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759885    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759893    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759911    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="prom-label-proxy"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759920    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="prom-label-proxy"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759934    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="prometheus"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759943    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="prometheus"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759961    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="init-config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759971    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="init-config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759980    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="alertmanager"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.759989    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="alertmanager"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760012    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="thanos-sidecar"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760020    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="thanos-sidecar"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760035    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-web"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760043    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-web"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760059    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-metric"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760068    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-metric"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760080    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" containerName="console"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760094    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" containerName="console"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760104    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="init-config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760112    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="init-config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760121    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-thanos"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760129    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-thanos"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760250    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="thanos-sidecar"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760268    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="config-reloader"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760280    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="prom-label-proxy"
Sep 11 11:54:36.762800 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760297    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="config-reloader"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760308    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa7e598f-7a5e-4fdc-8d45-68f6d3f7d756" containerName="console"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760316    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bbdbf3f-abe4-4f0c-830f-a47bb729254a" containerName="console"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760331    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-thanos"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760339    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760355    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-web"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760364    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760373    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="kube-rbac-proxy-web"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760388    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="kube-rbac-proxy-metric"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760397    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" containerName="prometheus"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.760406    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e78e29-0d59-4a4c-ad37-11df77d4754a" containerName="alertmanager"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:54:36.763048    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": container with ID starting with e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16 not found: ID does not exist" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.763081    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"} err="failed to get container status \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": rpc error: code = NotFound desc = could not find container \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": container with ID starting with e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16 not found: ID does not exist"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.763102    2586 scope.go:117] "RemoveContainer" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.763609    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"} err="failed to get container status \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": rpc error: code = NotFound desc = could not find container \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": container with ID starting with 8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4 not found: ID does not exist"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.763636    2586 scope.go:117] "RemoveContainer" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.763965    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"} err="failed to get container status \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": rpc error: code = NotFound desc = could not find container \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": container with ID starting with 67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d not found: ID does not exist"
Sep 11 11:54:36.764190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.763990    2586 scope.go:117] "RemoveContainer" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.764857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.764246    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"} err="failed to get container status \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": rpc error: code = NotFound desc = could not find container \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": container with ID starting with b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504 not found: ID does not exist"
Sep 11 11:54:36.764857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.764266    2586 scope.go:117] "RemoveContainer" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.764857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.764569    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"} err="failed to get container status \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": rpc error: code = NotFound desc = could not find container \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": container with ID starting with 5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b not found: ID does not exist"
Sep 11 11:54:36.764857 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.764600    2586 scope.go:117] "RemoveContainer" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.765043 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.764871    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"} err="failed to get container status \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": rpc error: code = NotFound desc = could not find container \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": container with ID starting with 4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82 not found: ID does not exist"
Sep 11 11:54:36.765043 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.764892    2586 scope.go:117] "RemoveContainer" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.765278 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.765235    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"} err="failed to get container status \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": rpc error: code = NotFound desc = could not find container \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": container with ID starting with 6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85 not found: ID does not exist"
Sep 11 11:54:36.765321 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.765296    2586 scope.go:117] "RemoveContainer" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.765617 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.765597    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"} err="failed to get container status \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": rpc error: code = NotFound desc = could not find container \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": container with ID starting with e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16 not found: ID does not exist"
Sep 11 11:54:36.765617 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.765616    2586 scope.go:117] "RemoveContainer" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.765761 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.765701    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.765817 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.765794    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"} err="failed to get container status \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": rpc error: code = NotFound desc = could not find container \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": container with ID starting with 8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4 not found: ID does not exist"
Sep 11 11:54:36.765817 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.765806    2586 scope.go:117] "RemoveContainer" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.766023 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766004    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"} err="failed to get container status \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": rpc error: code = NotFound desc = could not find container \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": container with ID starting with 67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d not found: ID does not exist"
Sep 11 11:54:36.766094 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766032    2586 scope.go:117] "RemoveContainer" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.766378 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766350    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"} err="failed to get container status \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": rpc error: code = NotFound desc = could not find container \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": container with ID starting with b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504 not found: ID does not exist"
Sep 11 11:54:36.766448 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766380    2586 scope.go:117] "RemoveContainer" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.766676 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766657    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"} err="failed to get container status \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": rpc error: code = NotFound desc = could not find container \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": container with ID starting with 5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b not found: ID does not exist"
Sep 11 11:54:36.766676 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766676    2586 scope.go:117] "RemoveContainer" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.766933 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766915    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"} err="failed to get container status \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": rpc error: code = NotFound desc = could not find container \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": container with ID starting with 4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82 not found: ID does not exist"
Sep 11 11:54:36.766991 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.766935    2586 scope.go:117] "RemoveContainer" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.767198 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.767182    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"} err="failed to get container status \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": rpc error: code = NotFound desc = could not find container \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": container with ID starting with 6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85 not found: ID does not exist"
Sep 11 11:54:36.767198 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.767198    2586 scope.go:117] "RemoveContainer" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.767474 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.767454    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"} err="failed to get container status \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": rpc error: code = NotFound desc = could not find container \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": container with ID starting with e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16 not found: ID does not exist"
Sep 11 11:54:36.767568 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.767524    2586 scope.go:117] "RemoveContainer" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.767789 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.767768    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"} err="failed to get container status \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": rpc error: code = NotFound desc = could not find container \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": container with ID starting with 8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4 not found: ID does not exist"
Sep 11 11:54:36.767789 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.767788    2586 scope.go:117] "RemoveContainer" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.768086 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768051    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"} err="failed to get container status \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": rpc error: code = NotFound desc = could not find container \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": container with ID starting with 67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d not found: ID does not exist"
Sep 11 11:54:36.768142 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768088    2586 scope.go:117] "RemoveContainer" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.768334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768310    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"} err="failed to get container status \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": rpc error: code = NotFound desc = could not find container \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": container with ID starting with b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504 not found: ID does not exist"
Sep 11 11:54:36.768334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768334    2586 scope.go:117] "RemoveContainer" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.768617 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768598    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"} err="failed to get container status \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": rpc error: code = NotFound desc = could not find container \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": container with ID starting with 5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b not found: ID does not exist"
Sep 11 11:54:36.768617 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768616    2586 scope.go:117] "RemoveContainer" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.768912 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768886    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"} err="failed to get container status \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": rpc error: code = NotFound desc = could not find container \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": container with ID starting with 4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82 not found: ID does not exist"
Sep 11 11:54:36.769010 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768921    2586 scope.go:117] "RemoveContainer" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.769010 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.768899    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 11:54:36.769127 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769050    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 11:54:36.769127 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769070    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 11:54:36.769226 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769135    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 11:54:36.769226 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769208    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"} err="failed to get container status \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": rpc error: code = NotFound desc = could not find container \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": container with ID starting with 6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85 not found: ID does not exist"
Sep 11 11:54:36.769318 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769232    2586 scope.go:117] "RemoveContainer" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.769454 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769437    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 11:54:36.769635 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769614    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"} err="failed to get container status \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": rpc error: code = NotFound desc = could not find container \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": container with ID starting with e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16 not found: ID does not exist"
Sep 11 11:54:36.769706 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769638    2586 scope.go:117] "RemoveContainer" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.769706 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769658    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 11:54:36.769802 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769779    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 11:54:36.769802 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769796    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7a433t7ffc7q7\""
Sep 11 11:54:36.769881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769801    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 11:54:36.769881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769857    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 11:54:36.770021 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.769991    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9zh5k\""
Sep 11 11:54:36.770079 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770006    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 11:54:36.770079 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770051    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 11:54:36.770079 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770004    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"} err="failed to get container status \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": rpc error: code = NotFound desc = could not find container \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": container with ID starting with 8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4 not found: ID does not exist"
Sep 11 11:54:36.770230 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770090    2586 scope.go:117] "RemoveContainer" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.770439 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770416    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"} err="failed to get container status \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": rpc error: code = NotFound desc = could not find container \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": container with ID starting with 67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d not found: ID does not exist"
Sep 11 11:54:36.770439 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770438    2586 scope.go:117] "RemoveContainer" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.770621 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770590    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 11:54:36.770795 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770754    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"} err="failed to get container status \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": rpc error: code = NotFound desc = could not find container \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": container with ID starting with b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504 not found: ID does not exist"
Sep 11 11:54:36.770795 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.770779    2586 scope.go:117] "RemoveContainer" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.771121 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.771100    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"} err="failed to get container status \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": rpc error: code = NotFound desc = could not find container \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": container with ID starting with 5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b not found: ID does not exist"
Sep 11 11:54:36.771190 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.771123    2586 scope.go:117] "RemoveContainer" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.771400 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.771366    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"} err="failed to get container status \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": rpc error: code = NotFound desc = could not find container \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": container with ID starting with 4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82 not found: ID does not exist"
Sep 11 11:54:36.771400 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.771394    2586 scope.go:117] "RemoveContainer" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.771769 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.771745    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"} err="failed to get container status \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": rpc error: code = NotFound desc = could not find container \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": container with ID starting with 6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85 not found: ID does not exist"
Sep 11 11:54:36.771769 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.771769    2586 scope.go:117] "RemoveContainer" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.772072 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.772044    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"} err="failed to get container status \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": rpc error: code = NotFound desc = could not find container \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": container with ID starting with e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16 not found: ID does not exist"
Sep 11 11:54:36.772154 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.772075    2586 scope.go:117] "RemoveContainer" containerID="8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"
Sep 11 11:54:36.772420 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.772395    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4"} err="failed to get container status \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": rpc error: code = NotFound desc = could not find container \"8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4\": container with ID starting with 8435a1a4c53fea22e176931fde793d3d7fc5ea9e41a6d5b32465cb70f71202e4 not found: ID does not exist"
Sep 11 11:54:36.772611 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.772422    2586 scope.go:117] "RemoveContainer" containerID="67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"
Sep 11 11:54:36.772745 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.772718    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d"} err="failed to get container status \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": rpc error: code = NotFound desc = could not find container \"67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d\": container with ID starting with 67a59478ea38fce32f3f8fe4982086905292ac52bb38cfa0559a2a5f695d1d2d not found: ID does not exist"
Sep 11 11:54:36.772808 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.772747    2586 scope.go:117] "RemoveContainer" containerID="b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"
Sep 11 11:54:36.773107 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.773070    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504"} err="failed to get container status \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": rpc error: code = NotFound desc = could not find container \"b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504\": container with ID starting with b9c23f5b2d5ec55c0b436dc37f92eae67402e9d784d5f6b91e4cc3a3dcb96504 not found: ID does not exist"
Sep 11 11:54:36.773187 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.773110    2586 scope.go:117] "RemoveContainer" containerID="5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"
Sep 11 11:54:36.773433 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.773264    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 11:54:36.773433 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.773417    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b"} err="failed to get container status \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": rpc error: code = NotFound desc = could not find container \"5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b\": container with ID starting with 5e77e8a87bc81e174ebca9622160baa4729ead58ce7fbfaba91195b9f392387b not found: ID does not exist"
Sep 11 11:54:36.773616 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.773438    2586 scope.go:117] "RemoveContainer" containerID="4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"
Sep 11 11:54:36.773911 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.773887    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82"} err="failed to get container status \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": rpc error: code = NotFound desc = could not find container \"4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82\": container with ID starting with 4946fe646fa7ff1329bf85b270f7b12386e049485cc2391272b290eb0720fe82 not found: ID does not exist"
Sep 11 11:54:36.774016 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.773913    2586 scope.go:117] "RemoveContainer" containerID="6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"
Sep 11 11:54:36.774393 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.774289    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85"} err="failed to get container status \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": rpc error: code = NotFound desc = could not find container \"6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85\": container with ID starting with 6093088381bea029e0ae2456346bbea2c24949753272474de37204dcd4b41b85 not found: ID does not exist"
Sep 11 11:54:36.774393 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.774316    2586 scope.go:117] "RemoveContainer" containerID="e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"
Sep 11 11:54:36.774660 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.774637    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16"} err="failed to get container status \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": rpc error: code = NotFound desc = could not find container \"e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16\": container with ID starting with e37eb528c45b0dd1c131d03997c9793694605cc58072c4c196962fd50200af16 not found: ID does not exist"
Sep 11 11:54:36.774998 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.774981    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 11:54:36.780343 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.780322    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:54:36.823026 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.822993    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f24b0d4-e719-40c2-9422-5c71b3d7b1a3" path="/var/lib/kubelet/pods/5f24b0d4-e719-40c2-9422-5c71b3d7b1a3/volumes"
Sep 11 11:54:36.881607 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881575    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.881743 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881629    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-config\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.881743 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881658    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.881743 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881687    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.881743 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881715    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.881743 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881739    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881769    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d9eb10-477e-4483-bb40-1412d7882842-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881801    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881827    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881852    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881878    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d9eb10-477e-4483-bb40-1412d7882842-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881900    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881931    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882002 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.881991    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882347 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.882057    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882347 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.882091    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882347 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.882132    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882347 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.882161    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882347 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.882193    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w27h\" (UniqueName: \"kubernetes.io/projected/83d9eb10-477e-4483-bb40-1412d7882842-kube-api-access-2w27h\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.882347 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.882223    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983382 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983327    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983382 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983368    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983684 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983387    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983684 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983415    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983684 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983479    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983684 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983536    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w27h\" (UniqueName: \"kubernetes.io/projected/83d9eb10-477e-4483-bb40-1412d7882842-kube-api-access-2w27h\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983684 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983570    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983684 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983610    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.983684 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983669    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-config\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983701    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983728    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983753    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983779    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983814    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d9eb10-477e-4483-bb40-1412d7882842-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983845    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983870    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983899    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983923    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d9eb10-477e-4483-bb40-1412d7882842-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983946    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.984041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.983977    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.986453 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.984935    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.986453 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.985594    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.986766 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.986743    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.986905    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-web-config\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987088    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987113    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987217    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987248    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987287    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987612    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987872    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.987995    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988099 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.988096    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988715 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.988690    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d9eb10-477e-4483-bb40-1412d7882842-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.988781 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.988699    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.989203 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.989178    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/83d9eb10-477e-4483-bb40-1412d7882842-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.989525 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.989471    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-config\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.989664 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.989646    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/83d9eb10-477e-4483-bb40-1412d7882842-config-out\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.989910 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.989894    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83d9eb10-477e-4483-bb40-1412d7882842-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:36.991454 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:36.991439    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w27h\" (UniqueName: \"kubernetes.io/projected/83d9eb10-477e-4483-bb40-1412d7882842-kube-api-access-2w27h\") pod \"prometheus-k8s-0\" (UID: \"83d9eb10-477e-4483-bb40-1412d7882842\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:37.076665 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:37.076633    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:37.353983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:37.353385    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:54:37.355897 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:54:37.355867    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d9eb10_477e_4483_bb40_1412d7882842.slice/crio-30cecdda653dde07ccf01ad48d691d331c33156b0cebb8ec1d0b703b79fe0df5 WatchSource:0}: Error finding container 30cecdda653dde07ccf01ad48d691d331c33156b0cebb8ec1d0b703b79fe0df5: Status 404 returned error can't find the container with id 30cecdda653dde07ccf01ad48d691d331c33156b0cebb8ec1d0b703b79fe0df5
Sep 11 11:54:37.708460 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:37.708387    2586 generic.go:356] "Generic (PLEG): container finished" podID="83d9eb10-477e-4483-bb40-1412d7882842" containerID="e3230563a83aa2a8364e5b3f6e1653b975bfbc895a4f50220fca3cac1eb7f5da" exitCode=0
Sep 11 11:54:37.708613 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:37.708472    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerDied","Data":"e3230563a83aa2a8364e5b3f6e1653b975bfbc895a4f50220fca3cac1eb7f5da"}
Sep 11 11:54:37.708613 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:37.708566    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerStarted","Data":"30cecdda653dde07ccf01ad48d691d331c33156b0cebb8ec1d0b703b79fe0df5"}
Sep 11 11:54:38.716948 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:38.716913    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerStarted","Data":"45f82c89ea12411c44f391505fe005d15808cc94fc216465a6b6cb37d37885d0"}
Sep 11 11:54:38.717269 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:38.716978    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerStarted","Data":"e17212c8cba10e3880d04e373f88ec90cf3b3fe20d6b9636edb974f521b2ded2"}
Sep 11 11:54:38.717269 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:38.717003    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerStarted","Data":"bd3232b773b39931ccae1e03772e927b09d930c1f844e3cd1a4d821b158b0eb2"}
Sep 11 11:54:38.717269 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:38.717027    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerStarted","Data":"f634d5235b1a228cd152b1a52f02b3cc7d69cdf0e21275c4327e259ab85ff7d5"}
Sep 11 11:54:38.717269 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:38.717049    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerStarted","Data":"0db3b79d64402f3574d4d1f76d948bf2ef88fa3036f0146d4b77125d4a567270"}
Sep 11 11:54:38.717269 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:38.717071    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"83d9eb10-477e-4483-bb40-1412d7882842","Type":"ContainerStarted","Data":"2eca655d43b6c15796f4acf63a83533f1a20ed41fc28d916d9bc4a978b0a3794"}
Sep 11 11:54:38.746559 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:38.746513    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.726950746 podStartE2EDuration="2.746478238s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.527492ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:54:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:54:38.743154432 +0000 UTC m=+172.453825781" watchObservedRunningTime="2026-09-11 11:54:38.746478238 +0000 UTC m=+172.457149576"
Sep 11 11:54:39.563092 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:39.563054    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:39.615985 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:39.615949    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:42.077657 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:42.077614    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:54:48.499731 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:48.499691    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.601077 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:48.601046    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:48.647623 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:48.647593    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:53.034404 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:53.034368    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:53.097768 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:53.097735    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:53.185381 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:53.185354    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:53.237844 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:53.237815    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:53.791532 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:53.791478    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:53.841943 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:53.841914    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:57.173087 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:54:57.173027    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:04.160428 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:04.160393    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:04.261427 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:04.261397    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:06.085328 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:06.085287    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:10.710576 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:10.710535    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:10.807567 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:10.807539    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:14.750680 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:14.750630    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:14.800544 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:14.800518    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:37.077115 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:37.077075    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:55:37.093733 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:37.093704    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:55:37.343107 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:37.343020    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:37.970450 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:37.970425    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:55:38.217904 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:38.217863    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:40.936685 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:40.936650    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:41.036504 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:41.036453    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:42.225716 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:42.225675    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:42.414334 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:42.414306    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:43.130903 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:43.130870    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:43.231193 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:43.231164    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:45.137425 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:45.137393    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:45.544211 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:45.544168    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:47.931119 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:47.931081    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:48.031252 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:48.031227    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:51.536689 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:51.536654    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:51.636285 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:51.636244    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:53.106890 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:53.106859    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:53.749509 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:53.748737    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:55.339049 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:55.339002    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:55.439652 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:55.439621    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:57.622261 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:57.622233    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:57.686281 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:57.686258    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:59.449767 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:55:59.449737    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:16.945638 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:16.945540    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:17.056155 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.056124    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:17.546049 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.545987    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:17.751899 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.751857    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:17.856801 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.856714    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:17.959745 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.959719    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"]
Sep 11 11:56:17.963436 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.963383    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:17.966540 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.966521    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 11:56:17.967538 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.967523    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 11:56:17.967872 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.967843    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 11:56:17.967967 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.967894    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 11:56:17.967967 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.967956    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ns5m5\""
Sep 11 11:56:17.983636 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:17.983617    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"]
Sep 11 11:56:18.088138 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.088113    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8f152b3-e789-4590-ab2a-d08d0423e646-manifests\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.088138 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.088140    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8f152b3-e789-4590-ab2a-d08d0423e646-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.088312 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.088157    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8f152b3-e789-4590-ab2a-d08d0423e646-tmp\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.088312 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.088256    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8f152b3-e789-4590-ab2a-d08d0423e646-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.088312 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.088293    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ntkh\" (UniqueName: \"kubernetes.io/projected/c8f152b3-e789-4590-ab2a-d08d0423e646-kube-api-access-2ntkh\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.128208 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.128136    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:18.188883 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.188858    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8f152b3-e789-4590-ab2a-d08d0423e646-manifests\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.188963 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.188885    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8f152b3-e789-4590-ab2a-d08d0423e646-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.188963 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.188903    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8f152b3-e789-4590-ab2a-d08d0423e646-tmp\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.188963 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.188956    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8f152b3-e789-4590-ab2a-d08d0423e646-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.189075 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.188982    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ntkh\" (UniqueName: \"kubernetes.io/projected/c8f152b3-e789-4590-ab2a-d08d0423e646-kube-api-access-2ntkh\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.189287 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.189265    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8f152b3-e789-4590-ab2a-d08d0423e646-manifests\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.191241 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.191217    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8f152b3-e789-4590-ab2a-d08d0423e646-tmp\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.191407 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.191388    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8f152b3-e789-4590-ab2a-d08d0423e646-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.191567 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.191549    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8f152b3-e789-4590-ab2a-d08d0423e646-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.197353 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.197324    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ntkh\" (UniqueName: \"kubernetes.io/projected/c8f152b3-e789-4590-ab2a-d08d0423e646-kube-api-access-2ntkh\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb\" (UID: \"c8f152b3-e789-4590-ab2a-d08d0423e646\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.225675 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.225643    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:18.276707 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.276687    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:18.329357 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.329325    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:18.426315 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.426245    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:18.524505 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.524457    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:18.550955 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:18.550926    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"]
Sep 11 11:56:18.553175 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:56:18.553148    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8f152b3_e789_4590_ab2a_d08d0423e646.slice/crio-50d529f001230a97708ad742f4ba3f2816c09a327f324dcc9988a30b3f80f839 WatchSource:0}: Error finding container 50d529f001230a97708ad742f4ba3f2816c09a327f324dcc9988a30b3f80f839: Status 404 returned error can't find the container with id 50d529f001230a97708ad742f4ba3f2816c09a327f324dcc9988a30b3f80f839
Sep 11 11:56:19.108136 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:19.108093    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb" event={"ID":"c8f152b3-e789-4590-ab2a-d08d0423e646","Type":"ContainerStarted","Data":"50d529f001230a97708ad742f4ba3f2816c09a327f324dcc9988a30b3f80f839"}
Sep 11 11:56:22.123134 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:22.123105    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb" event={"ID":"c8f152b3-e789-4590-ab2a-d08d0423e646","Type":"ContainerStarted","Data":"5aef15b31a1b558dc5bcc0c81d83757ea307c3fa567326eafc1494eebee5eb9e"}
Sep 11 11:56:23.128515 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:23.128459    2586 generic.go:356] "Generic (PLEG): container finished" podID="c8f152b3-e789-4590-ab2a-d08d0423e646" containerID="5aef15b31a1b558dc5bcc0c81d83757ea307c3fa567326eafc1494eebee5eb9e" exitCode=0
Sep 11 11:56:23.128886 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:23.128552    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb" event={"ID":"c8f152b3-e789-4590-ab2a-d08d0423e646","Type":"ContainerDied","Data":"5aef15b31a1b558dc5bcc0c81d83757ea307c3fa567326eafc1494eebee5eb9e"}
Sep 11 11:56:24.135582 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:24.135552    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb" event={"ID":"c8f152b3-e789-4590-ab2a-d08d0423e646","Type":"ContainerStarted","Data":"bb354ba0de2a42c4a1b7f68ef3e90405ddf03f84bd8a0f2e1359d779c293264d"}
Sep 11 11:56:24.135947 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:24.135722    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:24.157937 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:24.157892    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb" podStartSLOduration=2.980880763 podStartE2EDuration="7.157877312s" totalImagesPullingTime="3.171518956s" totalInitContainerRuntime="1.005477593s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:24.156378836 +0000 UTC m=+277.867050184" watchObservedRunningTime="2026-09-11 11:56:24.157877312 +0000 UTC m=+277.868548649"
Sep 11 11:56:29.767360 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:29.767287    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:29.871144 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:29.871111    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:30.153098 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:30.153071    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:30.254267 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:30.254241    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:35.142164 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:35.142133    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb"
Sep 11 11:56:36.912593 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:36.912553    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:37.038478 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:37.038445    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:41.554140 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.554109    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:41.624083 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.624060    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:41.627983 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.627956    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"]
Sep 11 11:56:41.631125 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.631109    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.634055 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.634018    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:56:41.634144 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.634070    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 11:56:41.636254 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.635892    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 11:56:41.638244 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.638223    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7dzh6\""
Sep 11 11:56:41.638457 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.638436    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 11:56:41.638925 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.638907    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 11:56:41.644041 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.644015    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"]
Sep 11 11:56:41.754609 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.754575    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwthv\" (UniqueName: \"kubernetes.io/projected/1388bfb1-b0b0-41e6-a151-4a7434f0437b-kube-api-access-dwthv\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.754794 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.754617    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1388bfb1-b0b0-41e6-a151-4a7434f0437b-cert\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.754794 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.754690    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1388bfb1-b0b0-41e6-a151-4a7434f0437b-metrics-cert\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.754794 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.754724    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1388bfb1-b0b0-41e6-a151-4a7434f0437b-manager-config\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.855465 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.855355    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1388bfb1-b0b0-41e6-a151-4a7434f0437b-metrics-cert\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.855465 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.855455    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1388bfb1-b0b0-41e6-a151-4a7434f0437b-manager-config\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.855707 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.855509    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwthv\" (UniqueName: \"kubernetes.io/projected/1388bfb1-b0b0-41e6-a151-4a7434f0437b-kube-api-access-dwthv\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.855707 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.855551    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1388bfb1-b0b0-41e6-a151-4a7434f0437b-cert\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.856344 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.856321    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1388bfb1-b0b0-41e6-a151-4a7434f0437b-manager-config\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.858373 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.858349    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1388bfb1-b0b0-41e6-a151-4a7434f0437b-metrics-cert\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.858603 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.858577    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1388bfb1-b0b0-41e6-a151-4a7434f0437b-cert\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.872852 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.872831    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwthv\" (UniqueName: \"kubernetes.io/projected/1388bfb1-b0b0-41e6-a151-4a7434f0437b-kube-api-access-dwthv\") pod \"lws-controller-manager-56c7cddbd4-kmkxj\" (UID: \"1388bfb1-b0b0-41e6-a151-4a7434f0437b\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:41.947092 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:41.947061    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:42.214428 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:42.214361    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"]
Sep 11 11:56:42.218591 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:56:42.218568    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1388bfb1_b0b0_41e6_a151_4a7434f0437b.slice/crio-41f53932a9bff5c71953ee2417f369f2958f67715db915e69b4d42e2730c9635 WatchSource:0}: Error finding container 41f53932a9bff5c71953ee2417f369f2958f67715db915e69b4d42e2730c9635: Status 404 returned error can't find the container with id 41f53932a9bff5c71953ee2417f369f2958f67715db915e69b4d42e2730c9635
Sep 11 11:56:43.216315 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:43.216273    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj" event={"ID":"1388bfb1-b0b0-41e6-a151-4a7434f0437b","Type":"ContainerStarted","Data":"41f53932a9bff5c71953ee2417f369f2958f67715db915e69b4d42e2730c9635"}
Sep 11 11:56:45.226638 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:45.226596    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj" event={"ID":"1388bfb1-b0b0-41e6-a151-4a7434f0437b","Type":"ContainerStarted","Data":"43bdecb138fdfc7d7fbd39d4eb2e66374495a1699fae7e9b088e6e61c18b27e1"}
Sep 11 11:56:45.227121 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:45.226729    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:45.243309 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:45.243256    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj" podStartSLOduration=1.939104653 podStartE2EDuration="4.243240378s" totalImagesPullingTime="2.304135725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:45.242428555 +0000 UTC m=+298.953099895" watchObservedRunningTime="2026-09-11 11:56:45.243240378 +0000 UTC m=+298.953911716"
Sep 11 11:56:46.738850 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:46.738818    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 11:56:46.741616 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:46.741593    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 11:56:46.747292 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:46.747270    2586 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 11:56:47.612057 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.612020    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:47.686371 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.686337    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"]
Sep 11 11:56:47.689787 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.689764    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.692561 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.692537    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l9tc8\""
Sep 11 11:56:47.692685 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.692622    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 11:56:47.692753 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.692540    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 11:56:47.700146 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.700125    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"]
Sep 11 11:56:47.806363 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.806336    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70f06c0-3711-470f-bd0a-deaf9877361f-tls-certs\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.806751 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.806388    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b70f06c0-3711-470f-bd0a-deaf9877361f-tmp\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.806751 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.806462    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxf2s\" (UniqueName: \"kubernetes.io/projected/b70f06c0-3711-470f-bd0a-deaf9877361f-kube-api-access-jxf2s\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.907460 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.907385    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70f06c0-3711-470f-bd0a-deaf9877361f-tls-certs\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.907460 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.907445    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b70f06c0-3711-470f-bd0a-deaf9877361f-tmp\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.907719 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.907469    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxf2s\" (UniqueName: \"kubernetes.io/projected/b70f06c0-3711-470f-bd0a-deaf9877361f-kube-api-access-jxf2s\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.910186 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.910157    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b70f06c0-3711-470f-bd0a-deaf9877361f-tmp\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.910521 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.910477    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70f06c0-3711-470f-bd0a-deaf9877361f-tls-certs\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.917121 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.917092    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxf2s\" (UniqueName: \"kubernetes.io/projected/b70f06c0-3711-470f-bd0a-deaf9877361f-kube-api-access-jxf2s\") pod \"kube-auth-proxy-8478ddd598-tzln4\" (UID: \"b70f06c0-3711-470f-bd0a-deaf9877361f\") " pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:47.930844 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:47.930818    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.001550 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.001516    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"
Sep 11 11:56:48.029907 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.029849    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.278781 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.278713    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8478ddd598-tzln4"]
Sep 11 11:56:48.281725 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:56:48.281699    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70f06c0_3711_470f_bd0a_deaf9877361f.slice/crio-35904ed983be3b9f752cc2696716662eced726ab24b6caec7c1b720a74635b0f WatchSource:0}: Error finding container 35904ed983be3b9f752cc2696716662eced726ab24b6caec7c1b720a74635b0f: Status 404 returned error can't find the container with id 35904ed983be3b9f752cc2696716662eced726ab24b6caec7c1b720a74635b0f
Sep 11 11:56:48.283739 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.283711    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:56:48.337650 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.337598    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.439368 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.439334    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.730902 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.730868    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.789875 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.789838    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:48.897718 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:48.897678    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:49.248649 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:49.248578    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4" event={"ID":"b70f06c0-3711-470f-bd0a-deaf9877361f","Type":"ContainerStarted","Data":"35904ed983be3b9f752cc2696716662eced726ab24b6caec7c1b720a74635b0f"}
Sep 11 11:56:49.704863 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:49.704833    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:52.266088 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:52.266053    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4" event={"ID":"b70f06c0-3711-470f-bd0a-deaf9877361f","Type":"ContainerStarted","Data":"87c9eab99e037a28496ba4ba421bb9608ccde34de4068657a1d84fb545a994e4"}
Sep 11 11:56:52.284259 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:52.284204    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8478ddd598-tzln4" podStartSLOduration=2.36330036 podStartE2EDuration="5.284188738s" totalImagesPullingTime="2.920888378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:52.282423928 +0000 UTC m=+305.993095267" watchObservedRunningTime="2026-09-11 11:56:52.284188738 +0000 UTC m=+305.994860128"
Sep 11 11:56:55.550442 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:55.550409    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:56.234579 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:56.234550    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-kmkxj"
Sep 11 11:56:57.216268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:57.216229    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:58.217084 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:58.217047    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:58.308617 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:58.308582    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:58.402581 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:58.402540    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:58.502073 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:58.501982    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:58.636548 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:58.636505    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:58.839954 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:56:58.839928    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:49.046268 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:57:49.046180    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:49.135758 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:57:49.135721    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:07.150562 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:07.150529    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:09.562380 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:09.562339    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:09.948501 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:09.948414    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:11.226777 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.226735    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:11.327831 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.327803    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:11.452817 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.452789    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54c6bd499-cfw7b"]
Sep 11 11:58:11.455941 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.455922    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.458761 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.458740    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wrkjb\""
Sep 11 11:58:11.458854 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.458779    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 11:58:11.460098 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.460082    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 11:58:11.460203 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.460092    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 11:58:11.460203 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.460138    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 11:58:11.460203 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.460150    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 11:58:11.464640 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.464622    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 11:58:11.467633 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.467613    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54c6bd499-cfw7b"]
Sep 11 11:58:11.520298 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.520241    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:11.560718 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.560696    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-console-config\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.560797 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.560726    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc41508a-27b8-455f-96d2-49f922e972be-console-oauth-config\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.560797 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.560758    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tqv4\" (UniqueName: \"kubernetes.io/projected/cc41508a-27b8-455f-96d2-49f922e972be-kube-api-access-5tqv4\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.560797 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.560776    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-trusted-ca-bundle\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.560904 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.560816    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc41508a-27b8-455f-96d2-49f922e972be-console-serving-cert\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.560904 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.560839    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-service-ca\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.560965 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.560896    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-oauth-serving-cert\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.621073 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.621047    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:11.661603 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.661582    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tqv4\" (UniqueName: \"kubernetes.io/projected/cc41508a-27b8-455f-96d2-49f922e972be-kube-api-access-5tqv4\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.661705 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.661614    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-trusted-ca-bundle\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.661705 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.661635    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc41508a-27b8-455f-96d2-49f922e972be-console-serving-cert\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.661866 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.661847    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-service-ca\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.661995 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.661884    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-oauth-serving-cert\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.661995 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.661919    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-console-config\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.661995 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.661950    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc41508a-27b8-455f-96d2-49f922e972be-console-oauth-config\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.662631 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.662610    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-console-config\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.662728 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.662633    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-trusted-ca-bundle\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.662728 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.662633    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-oauth-serving-cert\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.662728 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.662700    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc41508a-27b8-455f-96d2-49f922e972be-service-ca\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.664380 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.664351    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc41508a-27b8-455f-96d2-49f922e972be-console-oauth-config\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.664457 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.664413    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc41508a-27b8-455f-96d2-49f922e972be-console-serving-cert\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.675935 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.675913    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tqv4\" (UniqueName: \"kubernetes.io/projected/cc41508a-27b8-455f-96d2-49f922e972be-kube-api-access-5tqv4\") pod \"console-54c6bd499-cfw7b\" (UID: \"cc41508a-27b8-455f-96d2-49f922e972be\") " pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.728922 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.728850    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:11.766048 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.765988    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:11.828901 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:11.828756    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:12.047725 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:12.047700    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54c6bd499-cfw7b"]
Sep 11 11:58:12.050100 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:58:12.050069    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc41508a_27b8_455f_96d2_49f922e972be.slice/crio-66dec243fd78af80c5aef628d04834a942dafbbf202eaf1e3330241e6e7c6450 WatchSource:0}: Error finding container 66dec243fd78af80c5aef628d04834a942dafbbf202eaf1e3330241e6e7c6450: Status 404 returned error can't find the container with id 66dec243fd78af80c5aef628d04834a942dafbbf202eaf1e3330241e6e7c6450
Sep 11 11:58:12.252613 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:12.252585    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:12.355792 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:12.355725    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:12.564181 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:12.564148    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:12.605020 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:12.604988    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c6bd499-cfw7b" event={"ID":"cc41508a-27b8-455f-96d2-49f922e972be","Type":"ContainerStarted","Data":"0d0a554da759a6a6e874431ae11b242f4ca7b10b9a296eccac38b6accee00e2f"}
Sep 11 11:58:12.605149 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:12.605046    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c6bd499-cfw7b" event={"ID":"cc41508a-27b8-455f-96d2-49f922e972be","Type":"ContainerStarted","Data":"66dec243fd78af80c5aef628d04834a942dafbbf202eaf1e3330241e6e7c6450"}
Sep 11 11:58:12.629003 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:12.628922    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54c6bd499-cfw7b" podStartSLOduration=1.6289003389999999 podStartE2EDuration="1.628900339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:12.627762034 +0000 UTC m=+386.338433373" watchObservedRunningTime="2026-09-11 11:58:12.628900339 +0000 UTC m=+386.339571681"
Sep 11 11:58:13.210765 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:13.210737    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:14.417407 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:14.417362    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:14.551073 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:14.551040    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:15.149429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:15.149400    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:15.400783 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:15.400706    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:15.554628 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:15.554599    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:15.663229 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:15.663152    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:20.758241 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:20.758200    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:20.851070 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:20.851034    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:21.766638 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:21.766605    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:21.767069 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:21.766697    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:21.771556 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:21.771533    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:22.651162 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:22.651136    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54c6bd499-cfw7b"
Sep 11 11:58:22.789934 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:22.789902    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:22.890406 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:22.890376    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:22.949970 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:22.949897    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:24.380941 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:24.380871    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:24.468812 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:24.468782    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:25.149227 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.149198    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"]
Sep 11 11:58:25.149760 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.149732    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:25.152783 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.152766    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.155729 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.155710    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 11:58:25.155830 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.155731    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 11:58:25.155830 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.155750    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 11:58:25.155830 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.155750    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kbk8h\""
Sep 11 11:58:25.157011 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.156997    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 11:58:25.159912 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.159891    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"]
Sep 11 11:58:25.163238 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.163207    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7835421c-171e-43fe-ac86-e9d57be651c2-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.163357 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.163334    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvgfw\" (UniqueName: \"kubernetes.io/projected/7835421c-171e-43fe-ac86-e9d57be651c2-kube-api-access-rvgfw\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.163432 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.163410    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835421c-171e-43fe-ac86-e9d57be651c2-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.264348 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.264327    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7835421c-171e-43fe-ac86-e9d57be651c2-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.264431 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.264377    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvgfw\" (UniqueName: \"kubernetes.io/projected/7835421c-171e-43fe-ac86-e9d57be651c2-kube-api-access-rvgfw\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.264539 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.264505    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835421c-171e-43fe-ac86-e9d57be651c2-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.264598 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:58:25.264580    2586 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 11 11:58:25.264664 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:58:25.264651    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7835421c-171e-43fe-ac86-e9d57be651c2-plugin-serving-cert podName:7835421c-171e-43fe-ac86-e9d57be651c2 nodeName:}" failed. No retries permitted until 2026-09-11 11:58:25.764634576 +0000 UTC m=+399.475305895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/7835421c-171e-43fe-ac86-e9d57be651c2-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-x89n9" (UID: "7835421c-171e-43fe-ac86-e9d57be651c2") : secret "plugin-serving-cert" not found
Sep 11 11:58:25.264970 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.264955    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7835421c-171e-43fe-ac86-e9d57be651c2-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.274874 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.274855    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvgfw\" (UniqueName: \"kubernetes.io/projected/7835421c-171e-43fe-ac86-e9d57be651c2-kube-api-access-rvgfw\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.325164 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.325137    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:25.768501 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.768455    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835421c-171e-43fe-ac86-e9d57be651c2-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:25.770999 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:25.770970    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835421c-171e-43fe-ac86-e9d57be651c2-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x89n9\" (UID: \"7835421c-171e-43fe-ac86-e9d57be651c2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:26.071008 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:26.070984    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"
Sep 11 11:58:26.347683 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:26.347660    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9"]
Sep 11 11:58:26.349843 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:58:26.349813    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7835421c_171e_43fe_ac86_e9d57be651c2.slice/crio-afbfdb05101ad999fdb8a888ccc4a61b14a7de50da8b4f1a7dc043a7b9874ffa WatchSource:0}: Error finding container afbfdb05101ad999fdb8a888ccc4a61b14a7de50da8b4f1a7dc043a7b9874ffa: Status 404 returned error can't find the container with id afbfdb05101ad999fdb8a888ccc4a61b14a7de50da8b4f1a7dc043a7b9874ffa
Sep 11 11:58:26.665394 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:26.665336    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9" event={"ID":"7835421c-171e-43fe-ac86-e9d57be651c2","Type":"ContainerStarted","Data":"afbfdb05101ad999fdb8a888ccc4a61b14a7de50da8b4f1a7dc043a7b9874ffa"}
Sep 11 11:58:26.814787 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:26.814761    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:26.870162 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:26.870121    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:31.690314 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:31.690268    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9" event={"ID":"7835421c-171e-43fe-ac86-e9d57be651c2","Type":"ContainerStarted","Data":"9bf9565135b4d750dcaeea926117a764d1825aa8623f7e2bff193007f0ba83ba"}
Sep 11 11:58:31.705864 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:31.705814    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x89n9" podStartSLOduration=2.324162921 podStartE2EDuration="6.70579986s" totalImagesPullingTime="4.381636939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:31.705192068 +0000 UTC m=+405.415863409" watchObservedRunningTime="2026-09-11 11:58:31.70579986 +0000 UTC m=+405.416471221"
Sep 11 11:58:33.298223 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:33.298189    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:33.395386 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:33.395356    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:33.451771 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:33.451741    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:34.875801 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:34.875770    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"]
Sep 11 11:58:34.879502 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:34.879460    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:34.882327 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:34.882308    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xbl99\""
Sep 11 11:58:34.891327 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:34.891307    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"]
Sep 11 11:58:34.943379 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:34.943353    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dnqj\" (UniqueName: \"kubernetes.io/projected/c7216563-e2b4-423c-a98a-ec5af04dd7f9-kube-api-access-6dnqj\") pod \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:34.943504 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:34.943386    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7216563-e2b4-423c-a98a-ec5af04dd7f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:35.044020 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.043994    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dnqj\" (UniqueName: \"kubernetes.io/projected/c7216563-e2b4-423c-a98a-ec5af04dd7f9-kube-api-access-6dnqj\") pod \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:35.044115 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.044028    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7216563-e2b4-423c-a98a-ec5af04dd7f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:35.044317 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.044301    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7216563-e2b4-423c-a98a-ec5af04dd7f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:35.059856 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.059838    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dnqj\" (UniqueName: \"kubernetes.io/projected/c7216563-e2b4-423c-a98a-ec5af04dd7f9-kube-api-access-6dnqj\") pod \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:35.188878 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.188831    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:35.467563 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.467540    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"]
Sep 11 11:58:35.469812 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:58:35.469772    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7216563_e2b4_423c_a98a_ec5af04dd7f9.slice/crio-5ba3e9da6bc77e2be064581fa5a10069afc29c4e88c53befaeb044390e62c34f WatchSource:0}: Error finding container 5ba3e9da6bc77e2be064581fa5a10069afc29c4e88c53befaeb044390e62c34f: Status 404 returned error can't find the container with id 5ba3e9da6bc77e2be064581fa5a10069afc29c4e88c53befaeb044390e62c34f
Sep 11 11:58:35.666737 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.666699    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"]
Sep 11 11:58:35.670854 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.670828    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"]
Sep 11 11:58:35.690609 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.690588    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"]
Sep 11 11:58:35.694995 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.694978    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:35.705915 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.705892    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"]
Sep 11 11:58:35.750036 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.749961    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d22d60-35fa-46a0-bf99-ff60f3c64984-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-5jr2j\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:35.750036 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.750005    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wdc6\" (UniqueName: \"kubernetes.io/projected/74d22d60-35fa-46a0-bf99-ff60f3c64984-kube-api-access-7wdc6\") pod \"kuadrant-operator-controller-manager-5f685c496f-5jr2j\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:35.850966 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.850933    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d22d60-35fa-46a0-bf99-ff60f3c64984-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-5jr2j\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:35.851114 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.850985    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wdc6\" (UniqueName: \"kubernetes.io/projected/74d22d60-35fa-46a0-bf99-ff60f3c64984-kube-api-access-7wdc6\") pod \"kuadrant-operator-controller-manager-5f685c496f-5jr2j\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:35.851295 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.851276    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d22d60-35fa-46a0-bf99-ff60f3c64984-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-5jr2j\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:35.859533 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:35.859510    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wdc6\" (UniqueName: \"kubernetes.io/projected/74d22d60-35fa-46a0-bf99-ff60f3c64984-kube-api-access-7wdc6\") pod \"kuadrant-operator-controller-manager-5f685c496f-5jr2j\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:36.009037 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:36.008984    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:36.302794 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:36.302635    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"]
Sep 11 11:58:36.305718 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:58:36.305676    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d22d60_35fa_46a0_bf99_ff60f3c64984.slice/crio-45b52e964c6a466678d77484fcc27b29ee7dad3812b0f1c7baf1b53c6190c014 WatchSource:0}: Error finding container 45b52e964c6a466678d77484fcc27b29ee7dad3812b0f1c7baf1b53c6190c014: Status 404 returned error can't find the container with id 45b52e964c6a466678d77484fcc27b29ee7dad3812b0f1c7baf1b53c6190c014
Sep 11 11:58:36.723118 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:36.722965    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j" event={"ID":"74d22d60-35fa-46a0-bf99-ff60f3c64984","Type":"ContainerStarted","Data":"45b52e964c6a466678d77484fcc27b29ee7dad3812b0f1c7baf1b53c6190c014"}
Sep 11 11:58:39.943827 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:39.943787    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:40.042781 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:40.042735    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:40.747577 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:40.747532    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j" event={"ID":"74d22d60-35fa-46a0-bf99-ff60f3c64984","Type":"ContainerStarted","Data":"60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d"}
Sep 11 11:58:40.747767 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:40.747598    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:40.749270 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:40.749246    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd" podUID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" containerName="manager" containerID="cri-o://5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb" gracePeriod=2
Sep 11 11:58:40.769131 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:40.769086    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j" podStartSLOduration=2.159023378 podStartE2EDuration="5.769072891s" totalImagesPullingTime="3.610049513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:40.766752694 +0000 UTC m=+414.477424032" watchObservedRunningTime="2026-09-11 11:58:40.769072891 +0000 UTC m=+414.479744229"
Sep 11 11:58:40.986987 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:40.986965    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:41.095834 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.095811    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dnqj\" (UniqueName: \"kubernetes.io/projected/c7216563-e2b4-423c-a98a-ec5af04dd7f9-kube-api-access-6dnqj\") pod \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") "
Sep 11 11:58:41.095947 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.095906    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7216563-e2b4-423c-a98a-ec5af04dd7f9-extensions-socket-volume\") pod \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\" (UID: \"c7216563-e2b4-423c-a98a-ec5af04dd7f9\") "
Sep 11 11:58:41.096142 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.096120    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7216563-e2b4-423c-a98a-ec5af04dd7f9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c7216563-e2b4-423c-a98a-ec5af04dd7f9" (UID: "c7216563-e2b4-423c-a98a-ec5af04dd7f9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:58:41.098054 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.098034    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7216563-e2b4-423c-a98a-ec5af04dd7f9-kube-api-access-6dnqj" (OuterVolumeSpecName: "kube-api-access-6dnqj") pod "c7216563-e2b4-423c-a98a-ec5af04dd7f9" (UID: "c7216563-e2b4-423c-a98a-ec5af04dd7f9"). InnerVolumeSpecName "kube-api-access-6dnqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:58:41.197361 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.197325    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dnqj\" (UniqueName: \"kubernetes.io/projected/c7216563-e2b4-423c-a98a-ec5af04dd7f9-kube-api-access-6dnqj\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:58:41.197361 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.197359    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7216563-e2b4-423c-a98a-ec5af04dd7f9-extensions-socket-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:58:41.754761 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.754722    2586 generic.go:356] "Generic (PLEG): container finished" podID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" containerID="5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb" exitCode=2
Sep 11 11:58:41.754761 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.754762    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:41.754999 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.754811    2586 scope.go:117] "RemoveContainer" containerID="5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb"
Sep 11 11:58:41.757545 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:58:41.757504    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:41.764382 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.764364    2586 scope.go:117] "RemoveContainer" containerID="5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb"
Sep 11 11:58:41.764705 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:58:41.764687    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb\": container with ID starting with 5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb not found: ID does not exist" containerID="5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb"
Sep 11 11:58:41.764769 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:41.764714    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb"} err="failed to get container status \"5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb\": rpc error: code = NotFound desc = could not find container \"5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb\": container with ID starting with 5ee732be348b8425d6bed5833d0a822d9a637bde00df67796047da81b575e3fb not found: ID does not exist"
Sep 11 11:58:41.765823 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:58:41.765799    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-d6gzd\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-d6gzd"
Sep 11 11:58:42.822359 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:42.822325    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" path="/var/lib/kubelet/pods/c7216563-e2b4-423c-a98a-ec5af04dd7f9/volumes"
Sep 11 11:58:43.729941 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:43.729912    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:43.829951 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:43.829919    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:43.967795 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:43.967769    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:44.069179 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:44.069154    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:47.535366 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:47.535340    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:47.635972 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:47.635946    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:51.757279 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:51.757252    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:58:52.782894 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:52.782856    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:52.884618 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:52.884580    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:53.311773 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:53.311746    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:53.465163 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:53.465132    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:53.559137 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:58:53.559107    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:03.913241 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:03.913203    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"]
Sep 11 11:59:03.913741 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:03.913533    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j" podUID="74d22d60-35fa-46a0-bf99-ff60f3c64984" containerName="manager" containerID="cri-o://60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d" gracePeriod=10
Sep 11 11:59:04.161676 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.161652    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:59:04.260509 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.260427    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d22d60-35fa-46a0-bf99-ff60f3c64984-extensions-socket-volume\") pod \"74d22d60-35fa-46a0-bf99-ff60f3c64984\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") "
Sep 11 11:59:04.260633 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.260543    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wdc6\" (UniqueName: \"kubernetes.io/projected/74d22d60-35fa-46a0-bf99-ff60f3c64984-kube-api-access-7wdc6\") pod \"74d22d60-35fa-46a0-bf99-ff60f3c64984\" (UID: \"74d22d60-35fa-46a0-bf99-ff60f3c64984\") "
Sep 11 11:59:04.260851 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.260827    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d22d60-35fa-46a0-bf99-ff60f3c64984-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "74d22d60-35fa-46a0-bf99-ff60f3c64984" (UID: "74d22d60-35fa-46a0-bf99-ff60f3c64984"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:59:04.262648 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.262626    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74d22d60-35fa-46a0-bf99-ff60f3c64984-kube-api-access-7wdc6" (OuterVolumeSpecName: "kube-api-access-7wdc6") pod "74d22d60-35fa-46a0-bf99-ff60f3c64984" (UID: "74d22d60-35fa-46a0-bf99-ff60f3c64984"). InnerVolumeSpecName "kube-api-access-7wdc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:04.361817 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.361798    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wdc6\" (UniqueName: \"kubernetes.io/projected/74d22d60-35fa-46a0-bf99-ff60f3c64984-kube-api-access-7wdc6\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:59:04.361817 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.361819    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d22d60-35fa-46a0-bf99-ff60f3c64984-extensions-socket-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:59:04.875296 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.875268    2586 generic.go:356] "Generic (PLEG): container finished" podID="74d22d60-35fa-46a0-bf99-ff60f3c64984" containerID="60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d" exitCode=0
Sep 11 11:59:04.875422 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.875311    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"
Sep 11 11:59:04.875422 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.875322    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j" event={"ID":"74d22d60-35fa-46a0-bf99-ff60f3c64984","Type":"ContainerDied","Data":"60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d"}
Sep 11 11:59:04.875422 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.875362    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j" event={"ID":"74d22d60-35fa-46a0-bf99-ff60f3c64984","Type":"ContainerDied","Data":"45b52e964c6a466678d77484fcc27b29ee7dad3812b0f1c7baf1b53c6190c014"}
Sep 11 11:59:04.875422 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.875384    2586 scope.go:117] "RemoveContainer" containerID="60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d"
Sep 11 11:59:04.884408 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.884372    2586 scope.go:117] "RemoveContainer" containerID="60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d"
Sep 11 11:59:04.884728 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:59:04.884706    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d\": container with ID starting with 60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d not found: ID does not exist" containerID="60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d"
Sep 11 11:59:04.884799 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.884738    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d"} err="failed to get container status \"60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d\": rpc error: code = NotFound desc = could not find container \"60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d\": container with ID starting with 60ba5aa9a1ce07f44b8f41283d42f049b85acd150def17566dba8a33c262406d not found: ID does not exist"
Sep 11 11:59:04.900970 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.900942    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"]
Sep 11 11:59:04.904912 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:04.904888    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-5jr2j"]
Sep 11 11:59:06.829685 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:06.829645    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74d22d60-35fa-46a0-bf99-ff60f3c64984" path="/var/lib/kubelet/pods/74d22d60-35fa-46a0-bf99-ff60f3c64984/volumes"
Sep 11 11:59:08.122586 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:08.122518    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:08.395006 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:08.394917    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:20.750408 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:20.750369    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:20.862285 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:20.862254    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:20.917325 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:20.917296    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:21.151220 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.151193    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 11:59:21.151532 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.151518    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" containerName="manager"
Sep 11 11:59:21.151596 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.151533    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" containerName="manager"
Sep 11 11:59:21.151596 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.151547    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74d22d60-35fa-46a0-bf99-ff60f3c64984" containerName="manager"
Sep 11 11:59:21.151596 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.151552    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="74d22d60-35fa-46a0-bf99-ff60f3c64984" containerName="manager"
Sep 11 11:59:21.151694 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.151607    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="74d22d60-35fa-46a0-bf99-ff60f3c64984" containerName="manager"
Sep 11 11:59:21.151694 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.151619    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7216563-e2b4-423c-a98a-ec5af04dd7f9" containerName="manager"
Sep 11 11:59:21.154675 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.154658    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.157609 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.157590    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 11:59:21.162864 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.162844    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 11:59:21.202897 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.202872    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 11:59:21.288981 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.288959    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pdxk\" (UniqueName: \"kubernetes.io/projected/decef843-bc3d-4301-b784-c41b671ba53c-kube-api-access-7pdxk\") pod \"limitador-limitador-84994f7ffc-xxwdg\" (UID: \"decef843-bc3d-4301-b784-c41b671ba53c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.289089 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.288991    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/decef843-bc3d-4301-b784-c41b671ba53c-config-file\") pod \"limitador-limitador-84994f7ffc-xxwdg\" (UID: \"decef843-bc3d-4301-b784-c41b671ba53c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.390198 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.390172    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pdxk\" (UniqueName: \"kubernetes.io/projected/decef843-bc3d-4301-b784-c41b671ba53c-kube-api-access-7pdxk\") pod \"limitador-limitador-84994f7ffc-xxwdg\" (UID: \"decef843-bc3d-4301-b784-c41b671ba53c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.390292 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.390203    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/decef843-bc3d-4301-b784-c41b671ba53c-config-file\") pod \"limitador-limitador-84994f7ffc-xxwdg\" (UID: \"decef843-bc3d-4301-b784-c41b671ba53c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.390799 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.390782    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/decef843-bc3d-4301-b784-c41b671ba53c-config-file\") pod \"limitador-limitador-84994f7ffc-xxwdg\" (UID: \"decef843-bc3d-4301-b784-c41b671ba53c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.398384 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.398363    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pdxk\" (UniqueName: \"kubernetes.io/projected/decef843-bc3d-4301-b784-c41b671ba53c-kube-api-access-7pdxk\") pod \"limitador-limitador-84994f7ffc-xxwdg\" (UID: \"decef843-bc3d-4301-b784-c41b671ba53c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.465240 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.465183    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:21.602112 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.602083    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:21.675069 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.675043    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:21.679123 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.679103    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-gkwgd"]
Sep 11 11:59:21.683452 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.683438    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gkwgd"
Sep 11 11:59:21.686332 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.686308    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t8brk\""
Sep 11 11:59:21.686821 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.686799    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gkwgd"]
Sep 11 11:59:21.735982 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.735961    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 11:59:21.738204 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:59:21.738184    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecef843_bc3d_4301_b784_c41b671ba53c.slice/crio-49968b868bb3dca89edae374bbdac1438d3ddf3887d18d66a1192677108675e2 WatchSource:0}: Error finding container 49968b868bb3dca89edae374bbdac1438d3ddf3887d18d66a1192677108675e2: Status 404 returned error can't find the container with id 49968b868bb3dca89edae374bbdac1438d3ddf3887d18d66a1192677108675e2
Sep 11 11:59:21.793771 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.793747    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6trkt\" (UniqueName: \"kubernetes.io/projected/4ce1745e-22e5-42e9-802e-cfaeac120d28-kube-api-access-6trkt\") pod \"authorino-59c974bd48-gkwgd\" (UID: \"4ce1745e-22e5-42e9-802e-cfaeac120d28\") " pod="kuadrant-system/authorino-59c974bd48-gkwgd"
Sep 11 11:59:21.894294 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.894272    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6trkt\" (UniqueName: \"kubernetes.io/projected/4ce1745e-22e5-42e9-802e-cfaeac120d28-kube-api-access-6trkt\") pod \"authorino-59c974bd48-gkwgd\" (UID: \"4ce1745e-22e5-42e9-802e-cfaeac120d28\") " pod="kuadrant-system/authorino-59c974bd48-gkwgd"
Sep 11 11:59:21.902628 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.902607    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6trkt\" (UniqueName: \"kubernetes.io/projected/4ce1745e-22e5-42e9-802e-cfaeac120d28-kube-api-access-6trkt\") pod \"authorino-59c974bd48-gkwgd\" (UID: \"4ce1745e-22e5-42e9-802e-cfaeac120d28\") " pod="kuadrant-system/authorino-59c974bd48-gkwgd"
Sep 11 11:59:21.953370 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.953348    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg" event={"ID":"decef843-bc3d-4301-b784-c41b671ba53c","Type":"ContainerStarted","Data":"49968b868bb3dca89edae374bbdac1438d3ddf3887d18d66a1192677108675e2"}
Sep 11 11:59:21.994098 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:21.994027    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gkwgd"
Sep 11 11:59:22.258565 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:22.258473    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gkwgd"]
Sep 11 11:59:22.261816 ip-10-0-130-2 kubenswrapper[2586]: W0911 11:59:22.261792    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ce1745e_22e5_42e9_802e_cfaeac120d28.slice/crio-e695319455bb156d0792e60135b1b47913295e3fdf2ae6319b1d91cc968a39a7 WatchSource:0}: Error finding container e695319455bb156d0792e60135b1b47913295e3fdf2ae6319b1d91cc968a39a7: Status 404 returned error can't find the container with id e695319455bb156d0792e60135b1b47913295e3fdf2ae6319b1d91cc968a39a7
Sep 11 11:59:22.964760 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:22.964717    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg" event={"ID":"decef843-bc3d-4301-b784-c41b671ba53c","Type":"ContainerStarted","Data":"2e0aecce63a91c998b8b4943501e4a78e40966112e95093e1ab5ad7f840b0f6f"}
Sep 11 11:59:22.965188 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:22.964791    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:22.966750 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:22.966710    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gkwgd" event={"ID":"4ce1745e-22e5-42e9-802e-cfaeac120d28","Type":"ContainerStarted","Data":"e695319455bb156d0792e60135b1b47913295e3fdf2ae6319b1d91cc968a39a7"}
Sep 11 11:59:22.981829 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:22.981746    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg" podStartSLOduration=1.002185628 podStartE2EDuration="1.981723843s" totalImagesPullingTime="979.538215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:22.980080066 +0000 UTC m=+456.690751405" watchObservedRunningTime="2026-09-11 11:59:22.981723843 +0000 UTC m=+456.692395186"
Sep 11 11:59:23.973104 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:23.973063    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gkwgd" event={"ID":"4ce1745e-22e5-42e9-802e-cfaeac120d28","Type":"ContainerStarted","Data":"9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37"}
Sep 11 11:59:23.989518 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:23.989446    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-gkwgd" podStartSLOduration=1.395248634 podStartE2EDuration="2.989428769s" totalImagesPullingTime="1.594180135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:23.987961841 +0000 UTC m=+457.698633186" watchObservedRunningTime="2026-09-11 11:59:23.989428769 +0000 UTC m=+457.700100113"
Sep 11 11:59:33.974429 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:33.974388    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"
Sep 11 11:59:36.307929 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:36.307893    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:36.400246 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:36.400189    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:36.756939 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:36.756859    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:48.622176 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:48.622126    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:48.726809 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:48.726769    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:51.220333 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:51.220301    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gkwgd"]
Sep 11 11:59:51.220776 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:51.220471    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-gkwgd" podUID="4ce1745e-22e5-42e9-802e-cfaeac120d28" containerName="authorino" containerID="cri-o://9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37" gracePeriod=30
Sep 11 11:59:51.464340 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:51.464318    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gkwgd"
Sep 11 11:59:51.512271 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:51.512218    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6trkt\" (UniqueName: \"kubernetes.io/projected/4ce1745e-22e5-42e9-802e-cfaeac120d28-kube-api-access-6trkt\") pod \"4ce1745e-22e5-42e9-802e-cfaeac120d28\" (UID: \"4ce1745e-22e5-42e9-802e-cfaeac120d28\") "
Sep 11 11:59:51.514361 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:51.514335    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ce1745e-22e5-42e9-802e-cfaeac120d28-kube-api-access-6trkt" (OuterVolumeSpecName: "kube-api-access-6trkt") pod "4ce1745e-22e5-42e9-802e-cfaeac120d28" (UID: "4ce1745e-22e5-42e9-802e-cfaeac120d28"). InnerVolumeSpecName "kube-api-access-6trkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:51.613307 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:51.613281    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6trkt\" (UniqueName: \"kubernetes.io/projected/4ce1745e-22e5-42e9-802e-cfaeac120d28-kube-api-access-6trkt\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 11:59:52.100832 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.100799    2586 generic.go:356] "Generic (PLEG): container finished" podID="4ce1745e-22e5-42e9-802e-cfaeac120d28" containerID="9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37" exitCode=0
Sep 11 11:59:52.101071 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.100854    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gkwgd"
Sep 11 11:59:52.101071 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.100877    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gkwgd" event={"ID":"4ce1745e-22e5-42e9-802e-cfaeac120d28","Type":"ContainerDied","Data":"9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37"}
Sep 11 11:59:52.101071 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.100916    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gkwgd" event={"ID":"4ce1745e-22e5-42e9-802e-cfaeac120d28","Type":"ContainerDied","Data":"e695319455bb156d0792e60135b1b47913295e3fdf2ae6319b1d91cc968a39a7"}
Sep 11 11:59:52.101071 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.100935    2586 scope.go:117] "RemoveContainer" containerID="9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37"
Sep 11 11:59:52.109555 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.109533    2586 scope.go:117] "RemoveContainer" containerID="9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37"
Sep 11 11:59:52.109855 ip-10-0-130-2 kubenswrapper[2586]: E0911 11:59:52.109832    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37\": container with ID starting with 9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37 not found: ID does not exist" containerID="9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37"
Sep 11 11:59:52.109921 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.109866    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37"} err="failed to get container status \"9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37\": rpc error: code = NotFound desc = could not find container \"9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37\": container with ID starting with 9296522b281d05a43af8931922c03801badeebba1a580189a0ad13522abe3d37 not found: ID does not exist"
Sep 11 11:59:52.121098 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.121060    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gkwgd"]
Sep 11 11:59:52.123173 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.123148    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gkwgd"]
Sep 11 11:59:52.822394 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:52.822368    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ce1745e-22e5-42e9-802e-cfaeac120d28" path="/var/lib/kubelet/pods/4ce1745e-22e5-42e9-802e-cfaeac120d28/volumes"
Sep 11 11:59:55.741881 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:55.741850    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:55.839450 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:55.839422    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:55.898103 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:55.898075    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:56.434221 ip-10-0-130-2 kubenswrapper[2586]: I0911 11:59:56.434192    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:00.001408 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:00.001376    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:00.098699 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:00.098672    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:31.599354 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:31.599316    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:31.696971 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:31.696926    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:33.105140 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.105112    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:33.216258 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.216224    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:33.763538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.763477    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:33.927802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.927772    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:00:33.933292 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.933260    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:33.940743 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.940720    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"]
Sep 11 12:00:33.941065 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.941053    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ce1745e-22e5-42e9-802e-cfaeac120d28" containerName="authorino"
Sep 11 12:00:33.941109 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.941067    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ce1745e-22e5-42e9-802e-cfaeac120d28" containerName="authorino"
Sep 11 12:00:33.941148 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.941117    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ce1745e-22e5-42e9-802e-cfaeac120d28" containerName="authorino"
Sep 11 12:00:33.944052 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.944034    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:33.946837 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.946818    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 12:00:33.947599 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.947584    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q4sl7\""
Sep 11 12:00:33.956618 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:33.956600    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"]
Sep 11 12:00:34.018583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.018528    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27fa3b8-75aa-4764-8995-f5eaf0240260-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-vtc56\" (UID: \"a27fa3b8-75aa-4764-8995-f5eaf0240260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:34.018583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.018565    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2sr2\" (UniqueName: \"kubernetes.io/projected/a27fa3b8-75aa-4764-8995-f5eaf0240260-kube-api-access-p2sr2\") pod \"payload-pre-processing-59fbc47bf8-vtc56\" (UID: \"a27fa3b8-75aa-4764-8995-f5eaf0240260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:34.119755 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.119731    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27fa3b8-75aa-4764-8995-f5eaf0240260-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-vtc56\" (UID: \"a27fa3b8-75aa-4764-8995-f5eaf0240260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:34.120048 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.119767    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2sr2\" (UniqueName: \"kubernetes.io/projected/a27fa3b8-75aa-4764-8995-f5eaf0240260-kube-api-access-p2sr2\") pod \"payload-pre-processing-59fbc47bf8-vtc56\" (UID: \"a27fa3b8-75aa-4764-8995-f5eaf0240260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:34.120378 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.120361    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27fa3b8-75aa-4764-8995-f5eaf0240260-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-vtc56\" (UID: \"a27fa3b8-75aa-4764-8995-f5eaf0240260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:34.128153 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.128131    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2sr2\" (UniqueName: \"kubernetes.io/projected/a27fa3b8-75aa-4764-8995-f5eaf0240260-kube-api-access-p2sr2\") pod \"payload-pre-processing-59fbc47bf8-vtc56\" (UID: \"a27fa3b8-75aa-4764-8995-f5eaf0240260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:34.254126 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.254104    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:34.447350 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.447317    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:34.522472 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:34.522444    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"]
Sep 11 12:00:34.525059 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:00:34.525033    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27fa3b8_75aa_4764_8995_f5eaf0240260.slice/crio-fc94bafa919870824b982aaf6249abce5bb9b992685089e294c8358e85ebc6e3 WatchSource:0}: Error finding container fc94bafa919870824b982aaf6249abce5bb9b992685089e294c8358e85ebc6e3: Status 404 returned error can't find the container with id fc94bafa919870824b982aaf6249abce5bb9b992685089e294c8358e85ebc6e3
Sep 11 12:00:35.298235 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:35.298184    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56" event={"ID":"a27fa3b8-75aa-4764-8995-f5eaf0240260","Type":"ContainerStarted","Data":"fc94bafa919870824b982aaf6249abce5bb9b992685089e294c8358e85ebc6e3"}
Sep 11 12:00:37.308822 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:37.308788    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56" event={"ID":"a27fa3b8-75aa-4764-8995-f5eaf0240260","Type":"ContainerStarted","Data":"3a675db4899d9ea8dbceed7035eec65f1fb0f4ee434671b3dc0c8b85fbe65431"}
Sep 11 12:00:37.309239 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:37.308869    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:37.332733 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:37.332686    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56" podStartSLOduration=2.15508702 podStartE2EDuration="4.332672603s" totalImagesPullingTime="2.177585583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:37.330945259 +0000 UTC m=+531.041616598" watchObservedRunningTime="2026-09-11 12:00:37.332672603 +0000 UTC m=+531.043343940"
Sep 11 12:00:44.566852 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:44.566768    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:44.666299 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:44.666268    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:48.315142 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:48.315112    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vtc56"
Sep 11 12:00:58.079748 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:00:58.079713    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:02.839197 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:02.839159    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:03.402295 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:03.402259    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:01:14.235443 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:14.235406    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:14.920100 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:14.920072    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:01:22.423957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.423919    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"]
Sep 11 12:01:22.432163 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.432129    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.436869 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.436835    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 12:01:22.437022 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.436954    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 12:01:22.437022 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.437011    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 12:01:22.438064 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.438049    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7x4n7\""
Sep 11 12:01:22.442702 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.442677    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"]
Sep 11 12:01:22.475584 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.475557    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:22.488400 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.488374    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.488540 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.488404    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgthp\" (UniqueName: \"kubernetes.io/projected/9da20359-b125-441f-9590-8619b726330f-kube-api-access-cgthp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.488540 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.488423    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.488540 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.488459    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9da20359-b125-441f-9590-8619b726330f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.488716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.488567    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.488716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.488634    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589367 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589345    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589380    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgthp\" (UniqueName: \"kubernetes.io/projected/9da20359-b125-441f-9590-8619b726330f-kube-api-access-cgthp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589400    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589446    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9da20359-b125-441f-9590-8619b726330f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589634 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589512    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589634 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589564    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589848 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589829    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589928 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589856    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.589928 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.589908    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.591878 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.591860    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9da20359-b125-441f-9590-8619b726330f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.592137 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.592115    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9da20359-b125-441f-9590-8619b726330f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.614338 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.614315    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgthp\" (UniqueName: \"kubernetes.io/projected/9da20359-b125-441f-9590-8619b726330f-kube-api-access-cgthp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9\" (UID: \"9da20359-b125-441f-9590-8619b726330f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:22.745305 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:22.745255    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:23.028560 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:23.028497    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"]
Sep 11 12:01:23.031588 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:01:23.031564    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da20359_b125_441f_9590_8619b726330f.slice/crio-8295e52f8bf0931fdaf1cccda937e98cda04a6e78036b12e7791bfed4fb45ea2 WatchSource:0}: Error finding container 8295e52f8bf0931fdaf1cccda937e98cda04a6e78036b12e7791bfed4fb45ea2: Status 404 returned error can't find the container with id 8295e52f8bf0931fdaf1cccda937e98cda04a6e78036b12e7791bfed4fb45ea2
Sep 11 12:01:23.433275 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:23.433239    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:01:23.527736 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:23.527664    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9" event={"ID":"9da20359-b125-441f-9590-8619b726330f","Type":"ContainerStarted","Data":"8295e52f8bf0931fdaf1cccda937e98cda04a6e78036b12e7791bfed4fb45ea2"}
Sep 11 12:01:23.786874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:23.786762    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:24.534149 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:24.534066    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9" event={"ID":"9da20359-b125-441f-9590-8619b726330f","Type":"ContainerStarted","Data":"db9cd796394dcedbdd660005c013bf0308932d0dd1ec3b34027061bb007503f9"}
Sep 11 12:01:24.559842 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:24.559789    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9" podStartSLOduration=1.5276194730000001 podStartE2EDuration="2.559770975s" totalImagesPullingTime="1.032151502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:01:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:01:24.558015883 +0000 UTC m=+578.268687221" watchObservedRunningTime="2026-09-11 12:01:24.559770975 +0000 UTC m=+578.270442312"
Sep 11 12:01:25.942406 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:25.942372    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:26.042438 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:26.042397    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:27.215187 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:27.215156    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:01:29.449891 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:29.449850    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:31.418478 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:31.418445    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:01:32.745623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:32.745580    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:32.746048 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:32.745656    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:32.758473 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:32.758446    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:33.587290 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:33.587265    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9"
Sep 11 12:01:42.874042 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:42.874004    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:43.733357 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:43.733317    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:43.831462 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:43.831415    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:44.615704 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:44.615672    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:01:46.771121 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:46.771090    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:01:46.772013 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:46.771985    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:01:55.436459 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.436420    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"]
Sep 11 12:01:55.443152 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.443135    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.448817 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.448796    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 12:01:55.452112 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.452089    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"]
Sep 11 12:01:55.481343 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.481315    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:55.536223 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.536195    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw2lp\" (UniqueName: \"kubernetes.io/projected/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-kube-api-access-pw2lp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.536361 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.536234    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.536361 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.536256    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.536361 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.536277    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.536361 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.536318    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.536361 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.536333    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637197 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637165    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637360 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637207    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637360 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637246    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637360 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637264    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637578 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637434    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw2lp\" (UniqueName: \"kubernetes.io/projected/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-kube-api-access-pw2lp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637578 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637515    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637690 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637653    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637731 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637706    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.637885 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.637861    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.639654 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.639635    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.639958 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.639940    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.648222 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.648193    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw2lp\" (UniqueName: \"kubernetes.io/projected/8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b-kube-api-access-pw2lp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w\" (UID: \"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:55.752540 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:55.752467    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:01:56.026791 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:56.026615    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"]
Sep 11 12:01:56.029063 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:01:56.029039    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cdd77ab_5abf_4d1a_8491_3d8d02f4c53b.slice/crio-70e7dd85d5e59068fb37830d08f61cd5acb90c96ffa48441c60dab974f87e2b0 WatchSource:0}: Error finding container 70e7dd85d5e59068fb37830d08f61cd5acb90c96ffa48441c60dab974f87e2b0: Status 404 returned error can't find the container with id 70e7dd85d5e59068fb37830d08f61cd5acb90c96ffa48441c60dab974f87e2b0
Sep 11 12:01:56.030287 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:56.030268    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:01:56.686102 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:56.686070    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w" event={"ID":"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b","Type":"ContainerStarted","Data":"65425f3e401c15f08cf3bb8cfa1dd93f82ea0ce701bb37d07a6f2ec12c249ad8"}
Sep 11 12:01:56.686480 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:56.686110    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w" event={"ID":"8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b","Type":"ContainerStarted","Data":"70e7dd85d5e59068fb37830d08f61cd5acb90c96ffa48441c60dab974f87e2b0"}
Sep 11 12:01:56.706583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:56.706541    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w" podStartSLOduration=1.513466712 podStartE2EDuration="1.706525786s" totalImagesPullingTime="193.059074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:01:56.705899452 +0000 UTC m=+610.416570791" watchObservedRunningTime="2026-09-11 12:01:56.706525786 +0000 UTC m=+610.417197124"
Sep 11 12:01:57.522070 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:01:57.522038    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:02:05.752578 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:02:05.752548    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:02:05.753051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:02:05.752594    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:02:05.764969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:02:05.764943    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:02:06.741588 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:02:06.741564    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w"
Sep 11 12:02:42.774320 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:02:42.774278    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:26.627392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.627358    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bb5846b4f-rsb2m"]
Sep 11 12:03:26.630570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.630552    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:26.634961 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.634935    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 12:03:26.635081 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.634971    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t8brk\""
Sep 11 12:03:26.639019 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.638994    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bb5846b4f-rsb2m"]
Sep 11 12:03:26.648070 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.648047    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cab8006-af4c-43a4-9f71-d015f2e4b60e-tls-cert\") pod \"authorino-7bb5846b4f-rsb2m\" (UID: \"3cab8006-af4c-43a4-9f71-d015f2e4b60e\") " pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:26.648192 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.648089    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwddr\" (UniqueName: \"kubernetes.io/projected/3cab8006-af4c-43a4-9f71-d015f2e4b60e-kube-api-access-mwddr\") pod \"authorino-7bb5846b4f-rsb2m\" (UID: \"3cab8006-af4c-43a4-9f71-d015f2e4b60e\") " pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:26.748989 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.748962    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cab8006-af4c-43a4-9f71-d015f2e4b60e-tls-cert\") pod \"authorino-7bb5846b4f-rsb2m\" (UID: \"3cab8006-af4c-43a4-9f71-d015f2e4b60e\") " pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:26.749093 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.748996    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwddr\" (UniqueName: \"kubernetes.io/projected/3cab8006-af4c-43a4-9f71-d015f2e4b60e-kube-api-access-mwddr\") pod \"authorino-7bb5846b4f-rsb2m\" (UID: \"3cab8006-af4c-43a4-9f71-d015f2e4b60e\") " pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:26.751555 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.751528    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cab8006-af4c-43a4-9f71-d015f2e4b60e-tls-cert\") pod \"authorino-7bb5846b4f-rsb2m\" (UID: \"3cab8006-af4c-43a4-9f71-d015f2e4b60e\") " pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:26.758254 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.758236    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwddr\" (UniqueName: \"kubernetes.io/projected/3cab8006-af4c-43a4-9f71-d015f2e4b60e-kube-api-access-mwddr\") pod \"authorino-7bb5846b4f-rsb2m\" (UID: \"3cab8006-af4c-43a4-9f71-d015f2e4b60e\") " pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:26.941091 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:26.941027    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb5846b4f-rsb2m"
Sep 11 12:03:27.215274 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:27.215242    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bb5846b4f-rsb2m"]
Sep 11 12:03:27.215919 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:03:27.215892    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cab8006_af4c_43a4_9f71_d015f2e4b60e.slice/crio-80123341c52cb61f8e136090295095f1b7bcc41f37a58cd02cdc28f7654a4049 WatchSource:0}: Error finding container 80123341c52cb61f8e136090295095f1b7bcc41f37a58cd02cdc28f7654a4049: Status 404 returned error can't find the container with id 80123341c52cb61f8e136090295095f1b7bcc41f37a58cd02cdc28f7654a4049
Sep 11 12:03:28.111829 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:28.111791    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb5846b4f-rsb2m" event={"ID":"3cab8006-af4c-43a4-9f71-d015f2e4b60e","Type":"ContainerStarted","Data":"1b46523a26122ce1ef9514802e9b6caf3b4750b0a437d177e3b71dbc19bc50fa"}
Sep 11 12:03:28.112181 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:28.111838    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb5846b4f-rsb2m" event={"ID":"3cab8006-af4c-43a4-9f71-d015f2e4b60e","Type":"ContainerStarted","Data":"80123341c52cb61f8e136090295095f1b7bcc41f37a58cd02cdc28f7654a4049"}
Sep 11 12:03:28.129915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:28.129873    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bb5846b4f-rsb2m" podStartSLOduration=1.410391582 podStartE2EDuration="2.129858342s" totalImagesPullingTime="719.46676ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:28.12773549 +0000 UTC m=+701.838406829" watchObservedRunningTime="2026-09-11 12:03:28.129858342 +0000 UTC m=+701.840529680"
Sep 11 12:03:28.650857 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:28.650813    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:28.746284 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:28.746254    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:29.711790 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:29.711747    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:29.816617 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:29.816581    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:45.362070 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:45.361986    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:45.462105 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:45.462074    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:45.718256 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:45.718179    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:45.821040 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:45.821009    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:46.861808 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:46.856844    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:46.956578 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:46.956535    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:47.285367 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:47.285299    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:49.044262 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:49.044232    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:49.113434 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:49.113402    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:03:49.145003 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:49.144968    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:49.698186 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:49.698150    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:49.798876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:49.798838    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:50.073855 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.073826    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:50.241108 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.241079    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:50.265184 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.265161    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7788-f8ebfa"]
Sep 11 12:03:50.268346 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.268285    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:50.271189 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.271170    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-sw5w8\""
Sep 11 12:03:50.283791 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.283763    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7788-f8ebfa"]
Sep 11 12:03:50.339729 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.339657    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrgw\" (UniqueName: \"kubernetes.io/projected/6cb32487-e7ef-47d4-b027-a69f752d552a-kube-api-access-twrgw\") pod \"test-curl-7788-f8ebfa\" (UID: \"6cb32487-e7ef-47d4-b027-a69f752d552a\") " pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:50.341780 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.341756    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:50.400398 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.400374    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"]
Sep 11 12:03:50.403853 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.403835    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.406636 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.406618    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-34511f-dockercfg-s8hf8\""
Sep 11 12:03:50.406722 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.406705    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-34511f\""
Sep 11 12:03:50.415834 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.415814    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"]
Sep 11 12:03:50.440967 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.440942    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrgw\" (UniqueName: \"kubernetes.io/projected/6cb32487-e7ef-47d4-b027-a69f752d552a-kube-api-access-twrgw\") pod \"test-curl-7788-f8ebfa\" (UID: \"6cb32487-e7ef-47d4-b027-a69f752d552a\") " pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:50.453919 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.453898    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrgw\" (UniqueName: \"kubernetes.io/projected/6cb32487-e7ef-47d4-b027-a69f752d552a-kube-api-access-twrgw\") pod \"test-curl-7788-f8ebfa\" (UID: \"6cb32487-e7ef-47d4-b027-a69f752d552a\") " pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:50.541961 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.541933    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b9a1a6-7d81-41cf-89b2-1355778569ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.542060 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.541979    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdv5g\" (UniqueName: \"kubernetes.io/projected/b2b9a1a6-7d81-41cf-89b2-1355778569ae-kube-api-access-cdv5g\") pod \"payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.579889 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.579865    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:50.643193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.643155    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdv5g\" (UniqueName: \"kubernetes.io/projected/b2b9a1a6-7d81-41cf-89b2-1355778569ae-kube-api-access-cdv5g\") pod \"payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.643332 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.643314    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b9a1a6-7d81-41cf-89b2-1355778569ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.644595 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.644566    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b9a1a6-7d81-41cf-89b2-1355778569ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.652376 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.652356    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdv5g\" (UniqueName: \"kubernetes.io/projected/b2b9a1a6-7d81-41cf-89b2-1355778569ae-kube-api-access-cdv5g\") pod \"payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.713505 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.713466    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:50.851669 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.851644    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7788-f8ebfa"]
Sep 11 12:03:50.854059 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:03:50.854034    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cb32487_e7ef_47d4_b027_a69f752d552a.slice/crio-33ffdfcaa18c5c77c86e7a0b4d58e00632f5bfa97b449bd56767b9d445a2bcb4 WatchSource:0}: Error finding container 33ffdfcaa18c5c77c86e7a0b4d58e00632f5bfa97b449bd56767b9d445a2bcb4: Status 404 returned error can't find the container with id 33ffdfcaa18c5c77c86e7a0b4d58e00632f5bfa97b449bd56767b9d445a2bcb4
Sep 11 12:03:50.990815 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:50.990786    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"]
Sep 11 12:03:50.992781 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:03:50.992754    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b9a1a6_7d81_41cf_89b2_1355778569ae.slice/crio-1d1ded077b372bd34794b52c6d76b36ccdaa6b754266fc682dc8a7b0f25765df WatchSource:0}: Error finding container 1d1ded077b372bd34794b52c6d76b36ccdaa6b754266fc682dc8a7b0f25765df: Status 404 returned error can't find the container with id 1d1ded077b372bd34794b52c6d76b36ccdaa6b754266fc682dc8a7b0f25765df
Sep 11 12:03:51.230675 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:51.230594    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7788-f8ebfa" event={"ID":"6cb32487-e7ef-47d4-b027-a69f752d552a","Type":"ContainerStarted","Data":"33ffdfcaa18c5c77c86e7a0b4d58e00632f5bfa97b449bd56767b9d445a2bcb4"}
Sep 11 12:03:51.232185 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:51.232159    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8" event={"ID":"b2b9a1a6-7d81-41cf-89b2-1355778569ae","Type":"ContainerStarted","Data":"77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e"}
Sep 11 12:03:51.232320 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:51.232202    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8" event={"ID":"b2b9a1a6-7d81-41cf-89b2-1355778569ae","Type":"ContainerStarted","Data":"1d1ded077b372bd34794b52c6d76b36ccdaa6b754266fc682dc8a7b0f25765df"}
Sep 11 12:03:51.232320 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:51.232239    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:03:51.249280 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:51.249238    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8" podStartSLOduration=1.2492241179999999 podStartE2EDuration="1.249224118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:51.247610451 +0000 UTC m=+724.958281783" watchObservedRunningTime="2026-09-11 12:03:51.249224118 +0000 UTC m=+724.959895456"
Sep 11 12:03:53.191460 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.191388    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:53.243957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.243931    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7788-f8ebfa" event={"ID":"6cb32487-e7ef-47d4-b027-a69f752d552a","Type":"ContainerStarted","Data":"bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe"}
Sep 11 12:03:53.259331 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.259285    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7788-f8ebfa" podStartSLOduration=1.307547913 podStartE2EDuration="3.259271772s" totalImagesPullingTime="1.951723859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:53.25854271 +0000 UTC m=+726.969214049" watchObservedRunningTime="2026-09-11 12:03:53.259271772 +0000 UTC m=+726.969943109"
Sep 11 12:03:53.264956 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.264927    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:53.514054 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.513988    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:53.771209 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.771139    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:53.906163 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.906126    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:53.932131 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.932101    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7788-f8ebfa"]
Sep 11 12:03:53.938335 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:53.938314    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7788-f8ebfa"]
Sep 11 12:03:54.249353 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:03:54.249324    2586 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7788-f8ebfa" container="test-curl-7788-f8ebfa"
Sep 11 12:03:54.501194 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:54.501121    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:54.741778 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:54.741747    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:54.920776 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:54.920741    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:55.117089 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:55.117054    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:55.253555 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:55.253467    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7788-f8ebfa" podUID="6cb32487-e7ef-47d4-b027-a69f752d552a" containerName="test-curl-7788-f8ebfa" containerID="cri-o://bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe" gracePeriod=2
Sep 11 12:03:56.021476 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:56.021438    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:56.118999 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:56.118963    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:57.203100 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.203067    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:57.301688 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.301655    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:57.401712 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.401687    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:57.433704 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.433681    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:57.509569 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.509511    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twrgw\" (UniqueName: \"kubernetes.io/projected/6cb32487-e7ef-47d4-b027-a69f752d552a-kube-api-access-twrgw\") pod \"6cb32487-e7ef-47d4-b027-a69f752d552a\" (UID: \"6cb32487-e7ef-47d4-b027-a69f752d552a\") "
Sep 11 12:03:57.511622 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.511595    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cb32487-e7ef-47d4-b027-a69f752d552a-kube-api-access-twrgw" (OuterVolumeSpecName: "kube-api-access-twrgw") pod "6cb32487-e7ef-47d4-b027-a69f752d552a" (UID: "6cb32487-e7ef-47d4-b027-a69f752d552a"). InnerVolumeSpecName "kube-api-access-twrgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:03:57.606158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.606121    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:57.610279 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.610251    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twrgw\" (UniqueName: \"kubernetes.io/projected/6cb32487-e7ef-47d4-b027-a69f752d552a-kube-api-access-twrgw\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:03:57.709217 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.709193    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:57.895698 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.895667    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:57.990194 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:57.990163    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:58.177239 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.177175    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"]
Sep 11 12:03:58.177552 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.177539    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb32487-e7ef-47d4-b027-a69f752d552a" containerName="test-curl-7788-f8ebfa"
Sep 11 12:03:58.177608 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.177553    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb32487-e7ef-47d4-b027-a69f752d552a" containerName="test-curl-7788-f8ebfa"
Sep 11 12:03:58.177643 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.177611    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cb32487-e7ef-47d4-b027-a69f752d552a" containerName="test-curl-7788-f8ebfa"
Sep 11 12:03:58.180715 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.180698    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.183538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.183513    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a9aab05d\""
Sep 11 12:03:58.183632 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.183598    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a9aab05d-dockercfg-hdshc\""
Sep 11 12:03:58.193176 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.193155    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"]
Sep 11 12:03:58.269413 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.269386    2586 generic.go:356] "Generic (PLEG): container finished" podID="6cb32487-e7ef-47d4-b027-a69f752d552a" containerID="bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe" exitCode=137
Sep 11 12:03:58.269723 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.269421    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:58.269723 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.269505    2586 scope.go:117] "RemoveContainer" containerID="bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe"
Sep 11 12:03:58.277970 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.277956    2586 scope.go:117] "RemoveContainer" containerID="bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe"
Sep 11 12:03:58.278237 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:03:58.278218    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe\": container with ID starting with bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe not found: ID does not exist" containerID="bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe"
Sep 11 12:03:58.278290 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.278245    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe"} err="failed to get container status \"bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe\": rpc error: code = NotFound desc = could not find container \"bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe\": container with ID starting with bddc67bcc04c7ed6870d52ccdf6ab85626862aa565bd0d479d71b52449311dfe not found: ID does not exist"
Sep 11 12:03:58.279526 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:03:58.279473    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7788-f8ebfa\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="6cb32487-e7ef-47d4-b027-a69f752d552a" pod="opendatahub/test-curl-7788-f8ebfa"
Sep 11 12:03:58.315037 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.315011    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7q72\" (UniqueName: \"kubernetes.io/projected/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-kube-api-access-t7q72\") pod \"payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") " pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.315113 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.315088    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") " pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.415860 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.415835    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7q72\" (UniqueName: \"kubernetes.io/projected/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-kube-api-access-t7q72\") pod \"payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") " pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.415947 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.415898    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") " pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.416465 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.416449    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") " pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.423682 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.423660    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7q72\" (UniqueName: \"kubernetes.io/projected/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-kube-api-access-t7q72\") pod \"payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") " pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.490810 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.490760    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:58.765399 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.765322    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"]
Sep 11 12:03:58.768680 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:03:58.768653    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd0ed418_805a_47e7_ab3b_0a7725d8d0e7.slice/crio-1e26baeb9a4945edef7854289baafdef1c6688adb06683c17636a67774b2c8f1 WatchSource:0}: Error finding container 1e26baeb9a4945edef7854289baafdef1c6688adb06683c17636a67774b2c8f1: Status 404 returned error can't find the container with id 1e26baeb9a4945edef7854289baafdef1c6688adb06683c17636a67774b2c8f1
Sep 11 12:03:58.825712 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.825657    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cb32487-e7ef-47d4-b027-a69f752d552a" path="/var/lib/kubelet/pods/6cb32487-e7ef-47d4-b027-a69f752d552a/volumes"
Sep 11 12:03:58.881237 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.881206    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:58.979716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:58.979690    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:59.073112 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.073082    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:59.277227 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.277200    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h" event={"ID":"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7","Type":"ContainerStarted","Data":"f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a"}
Sep 11 12:03:59.277597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.277252    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h" event={"ID":"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7","Type":"ContainerStarted","Data":"1e26baeb9a4945edef7854289baafdef1c6688adb06683c17636a67774b2c8f1"}
Sep 11 12:03:59.277597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.277310    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:03:59.295595 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.295556    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h" podStartSLOduration=1.29554206 podStartE2EDuration="1.29554206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:59.294254982 +0000 UTC m=+733.004926321" watchObservedRunningTime="2026-09-11 12:03:59.29554206 +0000 UTC m=+733.006213399"
Sep 11 12:03:59.482554 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.482458    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:59.558941 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.558914    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"]
Sep 11 12:03:59.562296 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.562272    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:03:59.566339 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.566302    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-2c2847\""
Sep 11 12:03:59.566471 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.566403    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-2c2847-dockercfg-mtkdd\""
Sep 11 12:03:59.580002 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.579984    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"]
Sep 11 12:03:59.728464 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.728428    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2042335e-75f2-4cc6-a724-9044124748db-plugins-config-volume\") pod \"payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") " pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:03:59.728617 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.728556    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7pvq\" (UniqueName: \"kubernetes.io/projected/2042335e-75f2-4cc6-a724-9044124748db-kube-api-access-r7pvq\") pod \"payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") " pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:03:59.829127 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.829100    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7pvq\" (UniqueName: \"kubernetes.io/projected/2042335e-75f2-4cc6-a724-9044124748db-kube-api-access-r7pvq\") pod \"payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") " pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:03:59.829241 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.829153    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2042335e-75f2-4cc6-a724-9044124748db-plugins-config-volume\") pod \"payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") " pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:03:59.829796 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.829780    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2042335e-75f2-4cc6-a724-9044124748db-plugins-config-volume\") pod \"payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") " pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:03:59.837560 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.837538    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7pvq\" (UniqueName: \"kubernetes.io/projected/2042335e-75f2-4cc6-a724-9044124748db-kube-api-access-r7pvq\") pod \"payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") " pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:03:59.873277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:03:59.873255    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:04:00.147026 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.146946    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"]
Sep 11 12:04:00.149763 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:04:00.149736    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2042335e_75f2_4cc6_a724_9044124748db.slice/crio-c7fb1fa0ae62ba9c6a629e1ce149f0d6b45f07b7cea215c50b03e21d2e828ac8 WatchSource:0}: Error finding container c7fb1fa0ae62ba9c6a629e1ce149f0d6b45f07b7cea215c50b03e21d2e828ac8: Status 404 returned error can't find the container with id c7fb1fa0ae62ba9c6a629e1ce149f0d6b45f07b7cea215c50b03e21d2e828ac8
Sep 11 12:04:00.284424 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.284350    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd" event={"ID":"2042335e-75f2-4cc6-a724-9044124748db","Type":"ContainerStarted","Data":"2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8"}
Sep 11 12:04:00.284424 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.284414    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd" event={"ID":"2042335e-75f2-4cc6-a724-9044124748db","Type":"ContainerStarted","Data":"c7fb1fa0ae62ba9c6a629e1ce149f0d6b45f07b7cea215c50b03e21d2e828ac8"}
Sep 11 12:04:00.284924 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.284585    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:04:00.304433 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.304369    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd" podStartSLOduration=1.304349881 podStartE2EDuration="1.304349881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:00.303650588 +0000 UTC m=+734.014321928" watchObservedRunningTime="2026-09-11 12:04:00.304349881 +0000 UTC m=+734.015021220"
Sep 11 12:04:00.427073 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.426998    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:00.527705 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.527663    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:00.970209 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:00.970175    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:02.240213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:02.240178    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:04:04.711625 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:04.711593    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:04.769537 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:04.769505    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:04.900994 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:04.900960    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:04:07.185053 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:07.185018    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:08.594505 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.594450    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"]
Sep 11 12:04:08.597785 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.597762    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.600931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.600909    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-d2d09be7-kserve-self-signed-certs\""
Sep 11 12:04:08.607021 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.606995    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"]
Sep 11 12:04:08.701632 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.701602    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-model-cache\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.701713 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.701635    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b0df008-481f-487d-bf20-e2cddead6cf3-tls-certs\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.701713 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.701658    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-home\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.701802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.701724    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zglnf\" (UniqueName: \"kubernetes.io/projected/2b0df008-481f-487d-bf20-e2cddead6cf3-kube-api-access-zglnf\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.701802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.701758    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-tmp-dir\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.701802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.701784    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-dshm\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.802892 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.802863    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-model-cache\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.802995 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.802911    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b0df008-481f-487d-bf20-e2cddead6cf3-tls-certs\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.803057 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.803015    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-home\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.803057 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.803046    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zglnf\" (UniqueName: \"kubernetes.io/projected/2b0df008-481f-487d-bf20-e2cddead6cf3-kube-api-access-zglnf\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.803167 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.803085    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-tmp-dir\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.803240 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.803217    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-dshm\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.803298 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.803262    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-model-cache\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.803385 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.803368    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-home\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.803431 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.803392    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-tmp-dir\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.805424 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.805397    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-dshm\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.805620 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.805604    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b0df008-481f-487d-bf20-e2cddead6cf3-tls-certs\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.812527 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.812508    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zglnf\" (UniqueName: \"kubernetes.io/projected/2b0df008-481f-487d-bf20-e2cddead6cf3-kube-api-access-zglnf\") pod \"e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") " pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:08.908231 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:08.908173    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:09.184413 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:09.184389    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"]
Sep 11 12:04:09.186778 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:04:09.186746    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b0df008_481f_487d_bf20_e2cddead6cf3.slice/crio-1c8eb5af2034872c22ae52fc505a303b014c24fad3d8afe71927482955c4d762 WatchSource:0}: Error finding container 1c8eb5af2034872c22ae52fc505a303b014c24fad3d8afe71927482955c4d762: Status 404 returned error can't find the container with id 1c8eb5af2034872c22ae52fc505a303b014c24fad3d8afe71927482955c4d762
Sep 11 12:04:09.329520 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:09.329475    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2" event={"ID":"2b0df008-481f-487d-bf20-e2cddead6cf3","Type":"ContainerStarted","Data":"1f81c39d95cc3a8309a953317f9227fa614f85cd2ea1391e22229a461c6933d3"}
Sep 11 12:04:09.329679 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:09.329530    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2" event={"ID":"2b0df008-481f-487d-bf20-e2cddead6cf3","Type":"ContainerStarted","Data":"1c8eb5af2034872c22ae52fc505a303b014c24fad3d8afe71927482955c4d762"}
Sep 11 12:04:09.346912 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:09.346873    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2" podStartSLOduration=1.346857746 podStartE2EDuration="1.346857746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:09.346157132 +0000 UTC m=+743.056828469" watchObservedRunningTime="2026-09-11 12:04:09.346857746 +0000 UTC m=+743.057529083"
Sep 11 12:04:09.955924 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:09.955881    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:10.196384 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:10.196352    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:10.286424 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:10.286362    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:04:10.881794 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:10.881757    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:10.981917 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:10.981882    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:11.291912 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:11.291882    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:04:13.387454 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:13.387412    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:13.728722 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:13.728650    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:14.027674 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:14.027607    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:14.474671 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:14.474637    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:16.549294 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:16.549256    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:16.651433 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:16.651401    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:16.754530 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:16.754475    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:16.897031 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:16.896958    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:17.902961 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:17.902926    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.004183 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.004139    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.268355 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.268276    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.425192 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.425163    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.532343 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.532266    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.541667 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.541644    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"]
Sep 11 12:04:18.545314 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.545295    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.548475 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.548455    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e42660-dockercfg-9tpcg\""
Sep 11 12:04:18.548585 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.548455    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e42660\""
Sep 11 12:04:18.553368 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.553349    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"]
Sep 11 12:04:18.556650 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.556628    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.557537 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.557517    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"]
Sep 11 12:04:18.570275 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.570255    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"]
Sep 11 12:04:18.581897 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.581871    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nghbl\" (UniqueName: \"kubernetes.io/projected/40a093ff-8a4a-46a2-bb0d-efee9ef46768-kube-api-access-nghbl\") pod \"payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.581985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.581905    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqbgb\" (UniqueName: \"kubernetes.io/projected/fc98b0fc-b130-41e5-b3a0-03d41cb52373-kube-api-access-kqbgb\") pod \"payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.581985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.581926    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a093ff-8a4a-46a2-bb0d-efee9ef46768-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.581985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.581946    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc98b0fc-b130-41e5-b3a0-03d41cb52373-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.682801 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.682779    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nghbl\" (UniqueName: \"kubernetes.io/projected/40a093ff-8a4a-46a2-bb0d-efee9ef46768-kube-api-access-nghbl\") pod \"payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.682915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.682815    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqbgb\" (UniqueName: \"kubernetes.io/projected/fc98b0fc-b130-41e5-b3a0-03d41cb52373-kube-api-access-kqbgb\") pod \"payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.682915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.682845    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a093ff-8a4a-46a2-bb0d-efee9ef46768-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.682915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.682874    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc98b0fc-b130-41e5-b3a0-03d41cb52373-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.683558 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.683537    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a093ff-8a4a-46a2-bb0d-efee9ef46768-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.683636 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.683555    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc98b0fc-b130-41e5-b3a0-03d41cb52373-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.691372 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.691343    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqbgb\" (UniqueName: \"kubernetes.io/projected/fc98b0fc-b130-41e5-b3a0-03d41cb52373-kube-api-access-kqbgb\") pod \"payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.691458 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.691349    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nghbl\" (UniqueName: \"kubernetes.io/projected/40a093ff-8a4a-46a2-bb0d-efee9ef46768-kube-api-access-nghbl\") pod \"payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.856567 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.856540    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:18.867986 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.867968    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:18.872323 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.872301    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:18.909212 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.908280    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:18.909212 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.908363    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:18.928254 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:18.928227    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:19.135402 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.135378    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"]
Sep 11 12:04:19.137023 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:04:19.136996    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a093ff_8a4a_46a2_bb0d_efee9ef46768.slice/crio-19192dac6ac45338c6142dd6618c08214f20c64be4ba618a03934ae4e762cede WatchSource:0}: Error finding container 19192dac6ac45338c6142dd6618c08214f20c64be4ba618a03934ae4e762cede: Status 404 returned error can't find the container with id 19192dac6ac45338c6142dd6618c08214f20c64be4ba618a03934ae4e762cede
Sep 11 12:04:19.156091 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.156070    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"]
Sep 11 12:04:19.157622 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:04:19.157589    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc98b0fc_b130_41e5_b3a0_03d41cb52373.slice/crio-682b9265f52fae796b8e50d9425a5b4aeb861e09cec674a6ad1a11b4170c7659 WatchSource:0}: Error finding container 682b9265f52fae796b8e50d9425a5b4aeb861e09cec674a6ad1a11b4170c7659: Status 404 returned error can't find the container with id 682b9265f52fae796b8e50d9425a5b4aeb861e09cec674a6ad1a11b4170c7659
Sep 11 12:04:19.379687 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.379650    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9" event={"ID":"fc98b0fc-b130-41e5-b3a0-03d41cb52373","Type":"ContainerStarted","Data":"b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891"}
Sep 11 12:04:19.379847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.379791    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:19.379847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.379819    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9" event={"ID":"fc98b0fc-b130-41e5-b3a0-03d41cb52373","Type":"ContainerStarted","Data":"682b9265f52fae796b8e50d9425a5b4aeb861e09cec674a6ad1a11b4170c7659"}
Sep 11 12:04:19.381447 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.381419    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw" event={"ID":"40a093ff-8a4a-46a2-bb0d-efee9ef46768","Type":"ContainerStarted","Data":"604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae"}
Sep 11 12:04:19.381587 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.381475    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:19.381587 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.381526    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw" event={"ID":"40a093ff-8a4a-46a2-bb0d-efee9ef46768","Type":"ContainerStarted","Data":"19192dac6ac45338c6142dd6618c08214f20c64be4ba618a03934ae4e762cede"}
Sep 11 12:04:19.393996 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.393939    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:04:19.399477 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.399436    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9" podStartSLOduration=1.399421741 podStartE2EDuration="1.399421741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:19.397565273 +0000 UTC m=+753.108236626" watchObservedRunningTime="2026-09-11 12:04:19.399421741 +0000 UTC m=+753.110093078"
Sep 11 12:04:19.435096 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.435055    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw" podStartSLOduration=1.4350406740000001 podStartE2EDuration="1.435040674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:19.432765194 +0000 UTC m=+753.143436534" watchObservedRunningTime="2026-09-11 12:04:19.435040674 +0000 UTC m=+753.145712012"
Sep 11 12:04:19.690888 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:19.690805    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:20.388571 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:20.388540    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.079387 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:21.079357    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:21.857273 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:21.857242    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:22.390555 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:22.390519    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:22.683354 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:22.683274    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.268058 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:23.268027    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.367976 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:23.367942    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:25.148644 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:25.148605    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:28.001895 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:28.001863    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:04:30.389098 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:30.389068    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:04:30.389609 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:30.389166    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:04:37.473445 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:37.473380    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.098192 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.098159    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.124818 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.124781    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"]
Sep 11 12:04:38.125014 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.124990    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h" podUID="cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" containerName="payload-processing" containerID="cri-o://f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a" gracePeriod=30
Sep 11 12:04:38.198699 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.198671    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.299075 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.299018    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.379137 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.379117    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:04:38.402681 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.402654    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.449405 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.449377    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-plugins-config-volume\") pod \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") "
Sep 11 12:04:38.449574 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.449424    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7q72\" (UniqueName: \"kubernetes.io/projected/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-kube-api-access-t7q72\") pod \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\" (UID: \"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7\") "
Sep 11 12:04:38.449805 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.449786    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" (UID: "cd0ed418-805a-47e7-ab3b-0a7725d8d0e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:38.452507 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.452452    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-kube-api-access-t7q72" (OuterVolumeSpecName: "kube-api-access-t7q72") pod "cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" (UID: "cd0ed418-805a-47e7-ab3b-0a7725d8d0e7"). InnerVolumeSpecName "kube-api-access-t7q72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:38.466050 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.466005    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.477751 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.477726    2586 generic.go:356] "Generic (PLEG): container finished" podID="cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" containerID="f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a" exitCode=0
Sep 11 12:04:38.478028 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.477773    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"
Sep 11 12:04:38.478028 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.477813    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h" event={"ID":"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7","Type":"ContainerDied","Data":"f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a"}
Sep 11 12:04:38.478028 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.477865    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h" event={"ID":"cd0ed418-805a-47e7-ab3b-0a7725d8d0e7","Type":"ContainerDied","Data":"1e26baeb9a4945edef7854289baafdef1c6688adb06683c17636a67774b2c8f1"}
Sep 11 12:04:38.478028 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.477887    2586 scope.go:117] "RemoveContainer" containerID="f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a"
Sep 11 12:04:38.486849 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.486828    2586 scope.go:117] "RemoveContainer" containerID="f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a"
Sep 11 12:04:38.487093 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:04:38.487077    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a\": container with ID starting with f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a not found: ID does not exist" containerID="f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a"
Sep 11 12:04:38.487151 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.487100    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a"} err="failed to get container status \"f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a\": rpc error: code = NotFound desc = could not find container \"f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a\": container with ID starting with f9fbf803689521b8890e31b41b0ba88863aee48fd1b11accb1a46a5c1a61f15a not found: ID does not exist"
Sep 11 12:04:38.500226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.500202    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"]
Sep 11 12:04:38.506071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.506051    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9aab05d-7f477b6c49-2878h"]
Sep 11 12:04:38.550800 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.550778    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.550800 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.550798    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7q72\" (UniqueName: \"kubernetes.io/projected/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7-kube-api-access-t7q72\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.823346 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.823319    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" path="/var/lib/kubelet/pods/cd0ed418-805a-47e7-ab3b-0a7725d8d0e7/volumes"
Sep 11 12:04:38.882171 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.882138    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:38.955116 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:38.955086    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:39.097051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:39.096980    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:39.429245 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:39.429168    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:39.827709 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:39.827684    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:41.544582 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:41.544543    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.496410 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:42.496378    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.594685 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:42.594657    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.775513 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:42.775423    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.875647 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:42.875615    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.984598 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:42.984572    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:43.269995 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.269968    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:43.367726 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.367696    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:43.399728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.399704    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"]
Sep 11 12:04:43.400077 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.400065    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" containerName="payload-processing"
Sep 11 12:04:43.400122 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.400079    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" containerName="payload-processing"
Sep 11 12:04:43.400168 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.400159    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd0ed418-805a-47e7-ab3b-0a7725d8d0e7" containerName="payload-processing"
Sep 11 12:04:43.403244 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.403228    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.406342 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.406325    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-a6d3d15b\""
Sep 11 12:04:43.406422 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.406325    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-a6d3d15b-dockercfg-zfjk4\""
Sep 11 12:04:43.414126 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.414105    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"]
Sep 11 12:04:43.487358 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.487332    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e862e8-35a8-4d32-abac-7fed7a1d10ce-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.487452 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.487371    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh448\" (UniqueName: \"kubernetes.io/projected/73e862e8-35a8-4d32-abac-7fed7a1d10ce-kube-api-access-lh448\") pod \"payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.588214 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.588193    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh448\" (UniqueName: \"kubernetes.io/projected/73e862e8-35a8-4d32-abac-7fed7a1d10ce-kube-api-access-lh448\") pod \"payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.588343 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.588283    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e862e8-35a8-4d32-abac-7fed7a1d10ce-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.588905 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.588885    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e862e8-35a8-4d32-abac-7fed7a1d10ce-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.596529 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.596509    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh448\" (UniqueName: \"kubernetes.io/projected/73e862e8-35a8-4d32-abac-7fed7a1d10ce-kube-api-access-lh448\") pod \"payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.714034 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.714006    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:43.989473 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:43.989447    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"]
Sep 11 12:04:43.991581 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:04:43.991552    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e862e8_35a8_4d32_abac_7fed7a1d10ce.slice/crio-f8b8024061f97d36386e206dd72cc799c71d800ff9aed9cd9e0d80c8fb155e8e WatchSource:0}: Error finding container f8b8024061f97d36386e206dd72cc799c71d800ff9aed9cd9e0d80c8fb155e8e: Status 404 returned error can't find the container with id f8b8024061f97d36386e206dd72cc799c71d800ff9aed9cd9e0d80c8fb155e8e
Sep 11 12:04:44.509779 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:44.509744    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4" event={"ID":"73e862e8-35a8-4d32-abac-7fed7a1d10ce","Type":"ContainerStarted","Data":"6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a"}
Sep 11 12:04:44.509965 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:44.509801    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4" event={"ID":"73e862e8-35a8-4d32-abac-7fed7a1d10ce","Type":"ContainerStarted","Data":"f8b8024061f97d36386e206dd72cc799c71d800ff9aed9cd9e0d80c8fb155e8e"}
Sep 11 12:04:44.509965 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:44.509952    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:04:44.542398 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:44.542348    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4" podStartSLOduration=1.542333675 podStartE2EDuration="1.542333675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:44.54099157 +0000 UTC m=+778.251662910" watchObservedRunningTime="2026-09-11 12:04:44.542333675 +0000 UTC m=+778.253005013"
Sep 11 12:04:48.807779 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:48.807741    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:04:55.517226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:04:55.517196    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:05:08.600504 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:08.600399    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:05:13.515313 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.515265    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:13.543754 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.543718    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"]
Sep 11 12:05:13.544062 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.544031    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4" podUID="73e862e8-35a8-4d32-abac-7fed7a1d10ce" containerName="payload-processing" containerID="cri-o://6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a" gracePeriod=30
Sep 11 12:05:13.593522 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.593467    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:13.693923 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.693891    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:13.791478 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.791454    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:05:13.794093 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.794070    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:13.868123 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.868084    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:13.933839 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.933812    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e862e8-35a8-4d32-abac-7fed7a1d10ce-plugins-config-volume\") pod \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") "
Sep 11 12:05:13.933998 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.933885    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh448\" (UniqueName: \"kubernetes.io/projected/73e862e8-35a8-4d32-abac-7fed7a1d10ce-kube-api-access-lh448\") pod \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\" (UID: \"73e862e8-35a8-4d32-abac-7fed7a1d10ce\") "
Sep 11 12:05:13.934507 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.934170    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73e862e8-35a8-4d32-abac-7fed7a1d10ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73e862e8-35a8-4d32-abac-7fed7a1d10ce" (UID: "73e862e8-35a8-4d32-abac-7fed7a1d10ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:13.934507 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.934255    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73e862e8-35a8-4d32-abac-7fed7a1d10ce-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:05:13.937733 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.937675    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73e862e8-35a8-4d32-abac-7fed7a1d10ce-kube-api-access-lh448" (OuterVolumeSpecName: "kube-api-access-lh448") pod "73e862e8-35a8-4d32-abac-7fed7a1d10ce" (UID: "73e862e8-35a8-4d32-abac-7fed7a1d10ce"). InnerVolumeSpecName "kube-api-access-lh448". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:13.975659 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:13.975636    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:14.035423 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.035365    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh448\" (UniqueName: \"kubernetes.io/projected/73e862e8-35a8-4d32-abac-7fed7a1d10ce-kube-api-access-lh448\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:05:14.414716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.414682    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:14.657861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.657827    2586 generic.go:356] "Generic (PLEG): container finished" podID="73e862e8-35a8-4d32-abac-7fed7a1d10ce" containerID="6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a" exitCode=0
Sep 11 12:05:14.658324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.657891    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"
Sep 11 12:05:14.658324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.657916    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4" event={"ID":"73e862e8-35a8-4d32-abac-7fed7a1d10ce","Type":"ContainerDied","Data":"6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a"}
Sep 11 12:05:14.658324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.657980    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4" event={"ID":"73e862e8-35a8-4d32-abac-7fed7a1d10ce","Type":"ContainerDied","Data":"f8b8024061f97d36386e206dd72cc799c71d800ff9aed9cd9e0d80c8fb155e8e"}
Sep 11 12:05:14.658324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.658006    2586 scope.go:117] "RemoveContainer" containerID="6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a"
Sep 11 12:05:14.668317 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.668292    2586 scope.go:117] "RemoveContainer" containerID="6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a"
Sep 11 12:05:14.668667 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:05:14.668639    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a\": container with ID starting with 6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a not found: ID does not exist" containerID="6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a"
Sep 11 12:05:14.668780 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.668674    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a"} err="failed to get container status \"6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a\": rpc error: code = NotFound desc = could not find container \"6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a\": container with ID starting with 6ce34c9e40dec3c9d616827fe95766a83540cd63a69dcb345967efc310ad3d2a not found: ID does not exist"
Sep 11 12:05:14.684096 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.684073    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"]
Sep 11 12:05:14.690272 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.690251    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6d3d15b-5f869784cd-4pln4"]
Sep 11 12:05:14.791294 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.791239    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:14.824099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:14.824072    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73e862e8-35a8-4d32-abac-7fed7a1d10ce" path="/var/lib/kubelet/pods/73e862e8-35a8-4d32-abac-7fed7a1d10ce/volumes"
Sep 11 12:05:15.731105 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:15.731060    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:18.010863 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:18.010833    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:18.164969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:18.164941    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:18.883089 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:18.883058    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:18.984967 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:18.984941    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:19.083210 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.083180    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:19.455309 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.455282    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:19.597536 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.597507    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:19.754041 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.753968    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"]
Sep 11 12:05:19.754392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.754374    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e862e8-35a8-4d32-abac-7fed7a1d10ce" containerName="payload-processing"
Sep 11 12:05:19.754502 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.754395    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e862e8-35a8-4d32-abac-7fed7a1d10ce" containerName="payload-processing"
Sep 11 12:05:19.754568 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.754537    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="73e862e8-35a8-4d32-abac-7fed7a1d10ce" containerName="payload-processing"
Sep 11 12:05:19.757664 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.757644    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:19.760633 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.760616    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-24d087d2\""
Sep 11 12:05:19.760724 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.760631    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-24d087d2-dockercfg-j6f47\""
Sep 11 12:05:19.767583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.767558    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"]
Sep 11 12:05:19.886570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.886540    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06ae09a-d732-40f8-bc83-d5ff264b511c-plugins-config-volume\") pod \"payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") " pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:19.886683 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.886606    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndbzh\" (UniqueName: \"kubernetes.io/projected/b06ae09a-d732-40f8-bc83-d5ff264b511c-kube-api-access-ndbzh\") pod \"payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") " pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:19.987766 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.987725    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06ae09a-d732-40f8-bc83-d5ff264b511c-plugins-config-volume\") pod \"payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") " pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:19.987883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.987813    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndbzh\" (UniqueName: \"kubernetes.io/projected/b06ae09a-d732-40f8-bc83-d5ff264b511c-kube-api-access-ndbzh\") pod \"payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") " pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:19.988396 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.988377    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06ae09a-d732-40f8-bc83-d5ff264b511c-plugins-config-volume\") pod \"payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") " pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:19.997209 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:19.997178    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndbzh\" (UniqueName: \"kubernetes.io/projected/b06ae09a-d732-40f8-bc83-d5ff264b511c-kube-api-access-ndbzh\") pod \"payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") " pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:20.068493 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:20.068468    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:20.341863 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:20.341836    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"]
Sep 11 12:05:20.343956 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:05:20.343920    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06ae09a_d732_40f8_bc83_d5ff264b511c.slice/crio-5be218e714058ed28930bc80c5fe12afb77854a8a8d50ffc96a6f0879ff62035 WatchSource:0}: Error finding container 5be218e714058ed28930bc80c5fe12afb77854a8a8d50ffc96a6f0879ff62035: Status 404 returned error can't find the container with id 5be218e714058ed28930bc80c5fe12afb77854a8a8d50ffc96a6f0879ff62035
Sep 11 12:05:20.690932 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:20.690857    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27" event={"ID":"b06ae09a-d732-40f8-bc83-d5ff264b511c","Type":"ContainerStarted","Data":"3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565"}
Sep 11 12:05:20.690932 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:20.690910    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27" event={"ID":"b06ae09a-d732-40f8-bc83-d5ff264b511c","Type":"ContainerStarted","Data":"5be218e714058ed28930bc80c5fe12afb77854a8a8d50ffc96a6f0879ff62035"}
Sep 11 12:05:20.691105 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:20.690975    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:20.707557 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:20.707514    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27" podStartSLOduration=1.7075005989999998 podStartE2EDuration="1.707500599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:20.706601892 +0000 UTC m=+814.417273231" watchObservedRunningTime="2026-09-11 12:05:20.707500599 +0000 UTC m=+814.418171928"
Sep 11 12:05:23.994415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:23.994384    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:05:31.698177 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:31.698138    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:43.898516 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:43.898462    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:05:49.718382 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:49.718348    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:49.746355 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:49.746324    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"]
Sep 11 12:05:49.746597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:49.746571    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27" podUID="b06ae09a-d732-40f8-bc83-d5ff264b511c" containerName="payload-processing" containerID="cri-o://3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565" gracePeriod=30
Sep 11 12:05:49.819612 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:49.819578    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:49.922306 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:49.922246    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:49.986232 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:49.986207    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:49.990463 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:49.990444    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:50.011033 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.011010    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndbzh\" (UniqueName: \"kubernetes.io/projected/b06ae09a-d732-40f8-bc83-d5ff264b511c-kube-api-access-ndbzh\") pod \"b06ae09a-d732-40f8-bc83-d5ff264b511c\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") "
Sep 11 12:05:50.011113 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.011099    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06ae09a-d732-40f8-bc83-d5ff264b511c-plugins-config-volume\") pod \"b06ae09a-d732-40f8-bc83-d5ff264b511c\" (UID: \"b06ae09a-d732-40f8-bc83-d5ff264b511c\") "
Sep 11 12:05:50.011517 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.011463    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06ae09a-d732-40f8-bc83-d5ff264b511c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b06ae09a-d732-40f8-bc83-d5ff264b511c" (UID: "b06ae09a-d732-40f8-bc83-d5ff264b511c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:50.014314 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.013973    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b06ae09a-d732-40f8-bc83-d5ff264b511c-kube-api-access-ndbzh" (OuterVolumeSpecName: "kube-api-access-ndbzh") pod "b06ae09a-d732-40f8-bc83-d5ff264b511c" (UID: "b06ae09a-d732-40f8-bc83-d5ff264b511c"). InnerVolumeSpecName "kube-api-access-ndbzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:50.112102 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.112072    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06ae09a-d732-40f8-bc83-d5ff264b511c-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.112102 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.112100    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndbzh\" (UniqueName: \"kubernetes.io/projected/b06ae09a-d732-40f8-bc83-d5ff264b511c-kube-api-access-ndbzh\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.839666 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.839636    2586 generic.go:356] "Generic (PLEG): container finished" podID="b06ae09a-d732-40f8-bc83-d5ff264b511c" containerID="3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565" exitCode=0
Sep 11 12:05:50.840036 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.839671    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"
Sep 11 12:05:50.840036 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.839721    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27" event={"ID":"b06ae09a-d732-40f8-bc83-d5ff264b511c","Type":"ContainerDied","Data":"3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565"}
Sep 11 12:05:50.840036 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.839764    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27" event={"ID":"b06ae09a-d732-40f8-bc83-d5ff264b511c","Type":"ContainerDied","Data":"5be218e714058ed28930bc80c5fe12afb77854a8a8d50ffc96a6f0879ff62035"}
Sep 11 12:05:50.840036 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.839785    2586 scope.go:117] "RemoveContainer" containerID="3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565"
Sep 11 12:05:50.848406 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.848390    2586 scope.go:117] "RemoveContainer" containerID="3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565"
Sep 11 12:05:50.848668 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:05:50.848643    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565\": container with ID starting with 3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565 not found: ID does not exist" containerID="3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565"
Sep 11 12:05:50.848728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.848679    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565"} err="failed to get container status \"3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565\": rpc error: code = NotFound desc = could not find container \"3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565\": container with ID starting with 3c35bd87d08802af3e2eb0a4512c8e79bd29dbbd160abb3b3a866814a3e0f565 not found: ID does not exist"
Sep 11 12:05:50.857782 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.857759    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"]
Sep 11 12:05:50.862648 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:50.862627    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d087d2-8648c49b77-zhm27"]
Sep 11 12:05:51.058996 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:51.058955    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:51.444388 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:51.444358    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:51.817291 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:51.817260    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:52.593624 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:52.593582    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:52.823996 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:52.823964    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b06ae09a-d732-40f8-bc83-d5ff264b511c" path="/var/lib/kubelet/pods/b06ae09a-d732-40f8-bc83-d5ff264b511c/volumes"
Sep 11 12:05:54.811212 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:54.811175    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:54.903784 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:54.903752    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:55.959155 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:55.959128    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:56.025540 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.025504    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:56.133509 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.133456    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:56.451975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.451938    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:56.527879 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.527844    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:56.604385 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.604354    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"]
Sep 11 12:05:56.604805 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.604788    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b06ae09a-d732-40f8-bc83-d5ff264b511c" containerName="payload-processing"
Sep 11 12:05:56.604895 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.604807    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b06ae09a-d732-40f8-bc83-d5ff264b511c" containerName="payload-processing"
Sep 11 12:05:56.604949 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.604919    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b06ae09a-d732-40f8-bc83-d5ff264b511c" containerName="payload-processing"
Sep 11 12:05:56.609405 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.609385    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:56.613071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.613047    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-5e4884c8\""
Sep 11 12:05:56.614546 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.614529    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-5e4884c8-dockercfg-s7b8l\""
Sep 11 12:05:56.623998 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.623979    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"]
Sep 11 12:05:56.673093 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.673070    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ee7460c-bced-4438-a1e9-1f107317ed52-plugins-config-volume\") pod \"payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") " pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:56.673206 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.673113    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92mz9\" (UniqueName: \"kubernetes.io/projected/3ee7460c-bced-4438-a1e9-1f107317ed52-kube-api-access-92mz9\") pod \"payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") " pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:56.774104 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.774049    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ee7460c-bced-4438-a1e9-1f107317ed52-plugins-config-volume\") pod \"payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") " pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:56.774104 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.774086    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92mz9\" (UniqueName: \"kubernetes.io/projected/3ee7460c-bced-4438-a1e9-1f107317ed52-kube-api-access-92mz9\") pod \"payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") " pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:56.774709 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.774692    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ee7460c-bced-4438-a1e9-1f107317ed52-plugins-config-volume\") pod \"payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") " pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:56.798311 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.798290    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92mz9\" (UniqueName: \"kubernetes.io/projected/3ee7460c-bced-4438-a1e9-1f107317ed52-kube-api-access-92mz9\") pod \"payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") " pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:56.919479 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:56.919453    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:57.072039 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:57.072002    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:57.192230 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:57.192207    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"]
Sep 11 12:05:57.193977 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:05:57.193947    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee7460c_bced_4438_a1e9_1f107317ed52.slice/crio-41f45208883f44bc7e3c92e310f9a28325d1d36c745e88d2f4fb306610beabb8 WatchSource:0}: Error finding container 41f45208883f44bc7e3c92e310f9a28325d1d36c745e88d2f4fb306610beabb8: Status 404 returned error can't find the container with id 41f45208883f44bc7e3c92e310f9a28325d1d36c745e88d2f4fb306610beabb8
Sep 11 12:05:57.883851 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:57.883816    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6" event={"ID":"3ee7460c-bced-4438-a1e9-1f107317ed52","Type":"ContainerStarted","Data":"554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2"}
Sep 11 12:05:57.884027 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:57.883900    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6" event={"ID":"3ee7460c-bced-4438-a1e9-1f107317ed52","Type":"ContainerStarted","Data":"41f45208883f44bc7e3c92e310f9a28325d1d36c745e88d2f4fb306610beabb8"}
Sep 11 12:05:57.884027 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:57.883930    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:05:57.904373 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:57.904322    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6" podStartSLOduration=1.9043056219999999 podStartE2EDuration="1.904305622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:57.90395963 +0000 UTC m=+851.614630969" watchObservedRunningTime="2026-09-11 12:05:57.904305622 +0000 UTC m=+851.614976965"
Sep 11 12:05:57.944925 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:05:57.944901    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:01.607851 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:01.607812    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:06:02.085919 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.085887    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"]
Sep 11 12:06:02.089850 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.089826    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.092737 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.092707    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-d2d09be7-kserve-self-signed-certs\""
Sep 11 12:06:02.098078 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.098053    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"]
Sep 11 12:06:02.111885 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.111853    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:02.219119 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.219074    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-dshm\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.219308 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.219127    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqjqm\" (UniqueName: \"kubernetes.io/projected/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-kube-api-access-tqjqm\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.219308 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.219166    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-home\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.219308 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.219199    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tmp-dir\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.219308 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.219256    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tls-certs\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.219561 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.219317    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-model-cache\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320297 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320261    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-dshm\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320297 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320296    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqjqm\" (UniqueName: \"kubernetes.io/projected/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-kube-api-access-tqjqm\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320558 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320318    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-home\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320558 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320341    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tmp-dir\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320558 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320393    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tls-certs\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320558 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320444    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-model-cache\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320859 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320837    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-home\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320935 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320855    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-model-cache\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.320935 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.320835    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tmp-dir\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.322698 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.322680    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-dshm\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.322858 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.322840    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tls-certs\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.329867 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.329847    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqjqm\" (UniqueName: \"kubernetes.io/projected/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-kube-api-access-tqjqm\") pod \"e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") " pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.400474 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.400409    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:02.673775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.673672    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"]
Sep 11 12:06:02.676461 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:06:02.676435    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28c5afbd_4c59_40ce_bcd2_ac4b708705ce.slice/crio-52bd8e3d8936765a7a824357c0b8a9d33b1e889952d3ffc4d735d3dfa1440257 WatchSource:0}: Error finding container 52bd8e3d8936765a7a824357c0b8a9d33b1e889952d3ffc4d735d3dfa1440257: Status 404 returned error can't find the container with id 52bd8e3d8936765a7a824357c0b8a9d33b1e889952d3ffc4d735d3dfa1440257
Sep 11 12:06:02.909634 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.909601    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9" event={"ID":"28c5afbd-4c59-40ce-bcd2-ac4b708705ce","Type":"ContainerStarted","Data":"a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822"}
Sep 11 12:06:02.909787 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.909642    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9" event={"ID":"28c5afbd-4c59-40ce-bcd2-ac4b708705ce","Type":"ContainerStarted","Data":"52bd8e3d8936765a7a824357c0b8a9d33b1e889952d3ffc4d735d3dfa1440257"}
Sep 11 12:06:02.927889 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:02.927812    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9" podStartSLOduration=0.92779579 podStartE2EDuration="927.79579ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:02.926285071 +0000 UTC m=+856.636956422" watchObservedRunningTime="2026-09-11 12:06:02.92779579 +0000 UTC m=+856.638467127"
Sep 11 12:06:08.890997 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:08.890965    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:06:12.400885 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:12.400847    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:12.401409 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:12.400903    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:12.414046 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:12.414014    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:12.974329 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:12.974300    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:18.893669 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.893632    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"]
Sep 11 12:06:18.897166 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.897145    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:18.901772 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.901751    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-2c2847\"/\"openshift-service-ca.crt\""
Sep 11 12:06:18.901772 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.901765    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-2c2847\"/\"kube-root-ca.crt\""
Sep 11 12:06:18.901969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.901768    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-2c2847\"/\"autores-model-2c2847-kserve-self-signed-certs\""
Sep 11 12:06:18.901969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.901798    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-2c2847\"/\"default-dockercfg-4pjs5\""
Sep 11 12:06:18.906554 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.906533    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"]
Sep 11 12:06:18.932845 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:18.932787    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:19.058083 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.058063    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-tmp-dir\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.058189 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.058099    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lffr\" (UniqueName: \"kubernetes.io/projected/268a16cc-a264-4779-aa0e-f5e8e34162ed-kube-api-access-9lffr\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.058189 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.058139    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-dshm\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.058189 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.058179    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-model-cache\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.058297 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.058206    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-home\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.058297 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.058247    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268a16cc-a264-4779-aa0e-f5e8e34162ed-tls-certs\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159084 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159030    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-tmp-dir\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159084 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159066    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lffr\" (UniqueName: \"kubernetes.io/projected/268a16cc-a264-4779-aa0e-f5e8e34162ed-kube-api-access-9lffr\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159205 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159107    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-dshm\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159205 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159123    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-model-cache\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159205 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159144    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-home\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159205 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159170    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268a16cc-a264-4779-aa0e-f5e8e34162ed-tls-certs\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159472 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159450    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-tmp-dir\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159548 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159529    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-model-cache\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.159646 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.159629    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-home\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.162144 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.162121    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268a16cc-a264-4779-aa0e-f5e8e34162ed-tls-certs\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.162638 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.162621    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-dshm\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.167930 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.167911    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lffr\" (UniqueName: \"kubernetes.io/projected/268a16cc-a264-4779-aa0e-f5e8e34162ed-kube-api-access-9lffr\") pod \"autores-model-2c2847-kserve-67f474b6c5-n4j8v\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") " pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.209509 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.209465    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:19.481180 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.481156    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"]
Sep 11 12:06:19.482901 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:06:19.482876    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod268a16cc_a264_4779_aa0e_f5e8e34162ed.slice/crio-4eda48fbafc03b0ed2f0c809aa91942b47c3b16fb85b903299cca0b2c7c06b37 WatchSource:0}: Error finding container 4eda48fbafc03b0ed2f0c809aa91942b47c3b16fb85b903299cca0b2c7c06b37: Status 404 returned error can't find the container with id 4eda48fbafc03b0ed2f0c809aa91942b47c3b16fb85b903299cca0b2c7c06b37
Sep 11 12:06:19.996308 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.996271    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v" event={"ID":"268a16cc-a264-4779-aa0e-f5e8e34162ed","Type":"ContainerStarted","Data":"74b3642e2cdbcfd9ecb04188f92162b9bffc9e0a30dba15475f8cf2fe304dfe1"}
Sep 11 12:06:19.996696 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:19.996316    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v" event={"ID":"268a16cc-a264-4779-aa0e-f5e8e34162ed","Type":"ContainerStarted","Data":"4eda48fbafc03b0ed2f0c809aa91942b47c3b16fb85b903299cca0b2c7c06b37"}
Sep 11 12:06:20.014152 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:20.014107    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v" podStartSLOduration=2.01409283 podStartE2EDuration="2.01409283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:20.01375551 +0000 UTC m=+873.724426852" watchObservedRunningTime="2026-09-11 12:06:20.01409283 +0000 UTC m=+873.724764162"
Sep 11 12:06:22.601763 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:22.601725    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:06:26.480797 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.480764    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:26.512369 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.512338    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"]
Sep 11 12:06:26.512623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.512595    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6" podUID="3ee7460c-bced-4438-a1e9-1f107317ed52" containerName="payload-processing" containerID="cri-o://554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2" gracePeriod=30
Sep 11 12:06:26.582578 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.582546    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:26.681305 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.681272    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:26.735709 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.735616    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:26.759567 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.759549    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:06:26.821899 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.821876    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ee7460c-bced-4438-a1e9-1f107317ed52-plugins-config-volume\") pod \"3ee7460c-bced-4438-a1e9-1f107317ed52\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") "
Sep 11 12:06:26.822062 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.822041    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92mz9\" (UniqueName: \"kubernetes.io/projected/3ee7460c-bced-4438-a1e9-1f107317ed52-kube-api-access-92mz9\") pod \"3ee7460c-bced-4438-a1e9-1f107317ed52\" (UID: \"3ee7460c-bced-4438-a1e9-1f107317ed52\") "
Sep 11 12:06:26.822343 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.822315    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ee7460c-bced-4438-a1e9-1f107317ed52-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ee7460c-bced-4438-a1e9-1f107317ed52" (UID: "3ee7460c-bced-4438-a1e9-1f107317ed52"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:26.823909 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.823865    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:26.824694 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.824665    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee7460c-bced-4438-a1e9-1f107317ed52-kube-api-access-92mz9" (OuterVolumeSpecName: "kube-api-access-92mz9") pod "3ee7460c-bced-4438-a1e9-1f107317ed52" (UID: "3ee7460c-bced-4438-a1e9-1f107317ed52"). InnerVolumeSpecName "kube-api-access-92mz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:26.922980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.922955    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92mz9\" (UniqueName: \"kubernetes.io/projected/3ee7460c-bced-4438-a1e9-1f107317ed52-kube-api-access-92mz9\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:26.922980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:26.922975    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ee7460c-bced-4438-a1e9-1f107317ed52-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:27.028468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.028405    2586 generic.go:356] "Generic (PLEG): container finished" podID="3ee7460c-bced-4438-a1e9-1f107317ed52" containerID="554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2" exitCode=0
Sep 11 12:06:27.028468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.028456    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"
Sep 11 12:06:27.028640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.028515    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6" event={"ID":"3ee7460c-bced-4438-a1e9-1f107317ed52","Type":"ContainerDied","Data":"554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2"}
Sep 11 12:06:27.028640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.028592    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6" event={"ID":"3ee7460c-bced-4438-a1e9-1f107317ed52","Type":"ContainerDied","Data":"41f45208883f44bc7e3c92e310f9a28325d1d36c745e88d2f4fb306610beabb8"}
Sep 11 12:06:27.028640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.028615    2586 scope.go:117] "RemoveContainer" containerID="554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2"
Sep 11 12:06:27.037367 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.037340    2586 scope.go:117] "RemoveContainer" containerID="554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2"
Sep 11 12:06:27.037704 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:06:27.037684    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2\": container with ID starting with 554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2 not found: ID does not exist" containerID="554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2"
Sep 11 12:06:27.037780 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.037711    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2"} err="failed to get container status \"554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2\": rpc error: code = NotFound desc = could not find container \"554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2\": container with ID starting with 554abdc6bb6ecbbcd26e1ec10605aaa9dc992f08ddbcdcd099ef4fa8b8e210e2 not found: ID does not exist"
Sep 11 12:06:27.050314 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.050295    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"]
Sep 11 12:06:27.055726 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.055705    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5e4884c8-5d777447bf-wgvx6"]
Sep 11 12:06:27.482344 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.482318    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:27.570576 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.570547    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:27.673232 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:27.673209    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:28.824184 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:28.824149    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ee7460c-bced-4438-a1e9-1f107317ed52" path="/var/lib/kubelet/pods/3ee7460c-bced-4438-a1e9-1f107317ed52/volumes"
Sep 11 12:06:28.962758 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:28.962725    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:29.209960 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:29.209884    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:29.209960 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:29.209949    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:29.223006 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:29.222981    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:30.056209 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:30.056164    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:34.772937 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:34.772888    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:34.837912 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:34.837877    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:34.913937 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:34.913902    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:35.013681 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:35.013650    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:35.117417 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:35.117388    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:35.405754 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:35.405681    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:39.810443 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:39.810349    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:40.645936 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:40.645900    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:42.207957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:42.207929    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:06:42.914335 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:42.914302    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"]
Sep 11 12:06:42.934076 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:42.934040    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:43.479867 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:43.479832    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"]
Sep 11 12:06:43.498969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:43.498937    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:44.013604 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:44.013570    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"]
Sep 11 12:06:44.036367 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:44.036331    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:44.363420 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:44.363386    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:46.803649 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:46.803608    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:06:46.804839 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:46.804821    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:06:54.740858 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:54.740819    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:54.767606 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:54.767573    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"]
Sep 11 12:06:54.767893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:54.767861    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd" podUID="2042335e-75f2-4cc6-a724-9044124748db" containerName="payload-processing" containerID="cri-o://2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8" gracePeriod=30
Sep 11 12:06:54.841000 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:54.840968    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:54.937852 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:54.937664    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:55.000708 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.000625    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:55.022895 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.022872    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:06:55.149411 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.149387    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7pvq\" (UniqueName: \"kubernetes.io/projected/2042335e-75f2-4cc6-a724-9044124748db-kube-api-access-r7pvq\") pod \"2042335e-75f2-4cc6-a724-9044124748db\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") "
Sep 11 12:06:55.149563 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.149422    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2042335e-75f2-4cc6-a724-9044124748db-plugins-config-volume\") pod \"2042335e-75f2-4cc6-a724-9044124748db\" (UID: \"2042335e-75f2-4cc6-a724-9044124748db\") "
Sep 11 12:06:55.149802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.149772    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2042335e-75f2-4cc6-a724-9044124748db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2042335e-75f2-4cc6-a724-9044124748db" (UID: "2042335e-75f2-4cc6-a724-9044124748db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:55.151911 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.151888    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2042335e-75f2-4cc6-a724-9044124748db-kube-api-access-r7pvq" (OuterVolumeSpecName: "kube-api-access-r7pvq") pod "2042335e-75f2-4cc6-a724-9044124748db" (UID: "2042335e-75f2-4cc6-a724-9044124748db"). InnerVolumeSpecName "kube-api-access-r7pvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:55.166448 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.166424    2586 generic.go:356] "Generic (PLEG): container finished" podID="2042335e-75f2-4cc6-a724-9044124748db" containerID="2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8" exitCode=0
Sep 11 12:06:55.166561 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.166467    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"
Sep 11 12:06:55.166561 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.166470    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd" event={"ID":"2042335e-75f2-4cc6-a724-9044124748db","Type":"ContainerDied","Data":"2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8"}
Sep 11 12:06:55.166642 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.166560    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd" event={"ID":"2042335e-75f2-4cc6-a724-9044124748db","Type":"ContainerDied","Data":"c7fb1fa0ae62ba9c6a629e1ce149f0d6b45f07b7cea215c50b03e21d2e828ac8"}
Sep 11 12:06:55.166642 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.166589    2586 scope.go:117] "RemoveContainer" containerID="2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8"
Sep 11 12:06:55.175147 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.175131    2586 scope.go:117] "RemoveContainer" containerID="2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8"
Sep 11 12:06:55.175408 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:06:55.175388    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8\": container with ID starting with 2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8 not found: ID does not exist" containerID="2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8"
Sep 11 12:06:55.175459 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.175416    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8"} err="failed to get container status \"2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8\": rpc error: code = NotFound desc = could not find container \"2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8\": container with ID starting with 2eb57bbf3682ef8ec0d46dabdacf416494b23037d1a82b0219a73cac45446fc8 not found: ID does not exist"
Sep 11 12:06:55.188446 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.188421    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"]
Sep 11 12:06:55.194546 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.194518    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c2847-686dcbbfc-b4gdd"]
Sep 11 12:06:55.250324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.250297    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7pvq\" (UniqueName: \"kubernetes.io/projected/2042335e-75f2-4cc6-a724-9044124748db-kube-api-access-r7pvq\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.250324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.250320    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2042335e-75f2-4cc6-a724-9044124748db-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.271556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:55.271502    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:56.829807 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:56.829768    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2042335e-75f2-4cc6-a724-9044124748db" path="/var/lib/kubelet/pods/2042335e-75f2-4cc6-a724-9044124748db/volumes"
Sep 11 12:06:57.795567 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:57.795525    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:06:57.934393 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:57.934361    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2" podUID="2b0df008-481f-487d-bf20-e2cddead6cf3" containerName="main" containerID="cri-o://1f81c39d95cc3a8309a953317f9227fa614f85cd2ea1391e22229a461c6933d3" gracePeriod=45
Sep 11 12:06:58.183389 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.183361    2586 generic.go:356] "Generic (PLEG): container finished" podID="2b0df008-481f-487d-bf20-e2cddead6cf3" containerID="1f81c39d95cc3a8309a953317f9227fa614f85cd2ea1391e22229a461c6933d3" exitCode=0
Sep 11 12:06:58.183525 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.183425    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2" event={"ID":"2b0df008-481f-487d-bf20-e2cddead6cf3","Type":"ContainerDied","Data":"1f81c39d95cc3a8309a953317f9227fa614f85cd2ea1391e22229a461c6933d3"}
Sep 11 12:06:58.183525 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.183465    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2" event={"ID":"2b0df008-481f-487d-bf20-e2cddead6cf3","Type":"ContainerDied","Data":"1c8eb5af2034872c22ae52fc505a303b014c24fad3d8afe71927482955c4d762"}
Sep 11 12:06:58.183525 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.183476    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c8eb5af2034872c22ae52fc505a303b014c24fad3d8afe71927482955c4d762"
Sep 11 12:06:58.187861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.187844    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:06:58.276761 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.276730    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-model-cache\") pod \"2b0df008-481f-487d-bf20-e2cddead6cf3\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") "
Sep 11 12:06:58.276883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.276781    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zglnf\" (UniqueName: \"kubernetes.io/projected/2b0df008-481f-487d-bf20-e2cddead6cf3-kube-api-access-zglnf\") pod \"2b0df008-481f-487d-bf20-e2cddead6cf3\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") "
Sep 11 12:06:58.276883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.276815    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b0df008-481f-487d-bf20-e2cddead6cf3-tls-certs\") pod \"2b0df008-481f-487d-bf20-e2cddead6cf3\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") "
Sep 11 12:06:58.277003 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.276982    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b0df008-481f-487d-bf20-e2cddead6cf3" (UID: "2b0df008-481f-487d-bf20-e2cddead6cf3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.277065 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277013    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-home\") pod \"2b0df008-481f-487d-bf20-e2cddead6cf3\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") "
Sep 11 12:06:58.277065 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277055    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-tmp-dir\") pod \"2b0df008-481f-487d-bf20-e2cddead6cf3\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") "
Sep 11 12:06:58.277164 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277093    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-dshm\") pod \"2b0df008-481f-487d-bf20-e2cddead6cf3\" (UID: \"2b0df008-481f-487d-bf20-e2cddead6cf3\") "
Sep 11 12:06:58.277347 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277325    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-home" (OuterVolumeSpecName: "home") pod "2b0df008-481f-487d-bf20-e2cddead6cf3" (UID: "2b0df008-481f-487d-bf20-e2cddead6cf3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.277429 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277364    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b0df008-481f-487d-bf20-e2cddead6cf3" (UID: "2b0df008-481f-487d-bf20-e2cddead6cf3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.277508 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277449    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.277508 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277467    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.277508 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.277503    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.279186 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.279164    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b0df008-481f-487d-bf20-e2cddead6cf3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b0df008-481f-487d-bf20-e2cddead6cf3" (UID: "2b0df008-481f-487d-bf20-e2cddead6cf3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:58.279756 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.279738    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-dshm" (OuterVolumeSpecName: "dshm") pod "2b0df008-481f-487d-bf20-e2cddead6cf3" (UID: "2b0df008-481f-487d-bf20-e2cddead6cf3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.279832 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.279781    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b0df008-481f-487d-bf20-e2cddead6cf3-kube-api-access-zglnf" (OuterVolumeSpecName: "kube-api-access-zglnf") pod "2b0df008-481f-487d-bf20-e2cddead6cf3" (UID: "2b0df008-481f-487d-bf20-e2cddead6cf3"). InnerVolumeSpecName "kube-api-access-zglnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:58.378218 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.378167    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b0df008-481f-487d-bf20-e2cddead6cf3-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.378218 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.378186    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b0df008-481f-487d-bf20-e2cddead6cf3-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.378218 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.378195    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zglnf\" (UniqueName: \"kubernetes.io/projected/2b0df008-481f-487d-bf20-e2cddead6cf3-kube-api-access-zglnf\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.500439 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.500390    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9" podUID="28c5afbd-4c59-40ce-bcd2-ac4b708705ce" containerName="main" containerID="cri-o://a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822" gracePeriod=45
Sep 11 12:06:58.728278 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.728256    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:58.782314 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782286    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-model-cache\") pod \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") "
Sep 11 12:06:58.782314 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782317    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tmp-dir\") pod \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") "
Sep 11 12:06:58.782556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782344    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-home\") pod \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") "
Sep 11 12:06:58.782556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782388    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqjqm\" (UniqueName: \"kubernetes.io/projected/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-kube-api-access-tqjqm\") pod \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") "
Sep 11 12:06:58.782556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782407    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tls-certs\") pod \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") "
Sep 11 12:06:58.782556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782428    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-dshm\") pod \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\" (UID: \"28c5afbd-4c59-40ce-bcd2-ac4b708705ce\") "
Sep 11 12:06:58.782773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782606    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-model-cache" (OuterVolumeSpecName: "model-cache") pod "28c5afbd-4c59-40ce-bcd2-ac4b708705ce" (UID: "28c5afbd-4c59-40ce-bcd2-ac4b708705ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.782773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782627    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28c5afbd-4c59-40ce-bcd2-ac4b708705ce" (UID: "28c5afbd-4c59-40ce-bcd2-ac4b708705ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.782883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782773    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-home" (OuterVolumeSpecName: "home") pod "28c5afbd-4c59-40ce-bcd2-ac4b708705ce" (UID: "28c5afbd-4c59-40ce-bcd2-ac4b708705ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.782883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782872    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.782984 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782886    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.782984 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.782894    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.784671 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.784648    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-kube-api-access-tqjqm" (OuterVolumeSpecName: "kube-api-access-tqjqm") pod "28c5afbd-4c59-40ce-bcd2-ac4b708705ce" (UID: "28c5afbd-4c59-40ce-bcd2-ac4b708705ce"). InnerVolumeSpecName "kube-api-access-tqjqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:58.784773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.784670    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28c5afbd-4c59-40ce-bcd2-ac4b708705ce" (UID: "28c5afbd-4c59-40ce-bcd2-ac4b708705ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:58.784773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.784669    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-dshm" (OuterVolumeSpecName: "dshm") pod "28c5afbd-4c59-40ce-bcd2-ac4b708705ce" (UID: "28c5afbd-4c59-40ce-bcd2-ac4b708705ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:58.883933 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.883907    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqjqm\" (UniqueName: \"kubernetes.io/projected/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-kube-api-access-tqjqm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.884131 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.884107    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:58.884131 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:58.884131    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28c5afbd-4c59-40ce-bcd2-ac4b708705ce-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:59.035275 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.035223    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v" podUID="268a16cc-a264-4779-aa0e-f5e8e34162ed" containerName="main" containerID="cri-o://74b3642e2cdbcfd9ecb04188f92162b9bffc9e0a30dba15475f8cf2fe304dfe1" gracePeriod=45
Sep 11 12:06:59.190421 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.190376    2586 generic.go:356] "Generic (PLEG): container finished" podID="28c5afbd-4c59-40ce-bcd2-ac4b708705ce" containerID="a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822" exitCode=0
Sep 11 12:06:59.190421 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.190428    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"
Sep 11 12:06:59.190648 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.190455    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9" event={"ID":"28c5afbd-4c59-40ce-bcd2-ac4b708705ce","Type":"ContainerDied","Data":"a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822"}
Sep 11 12:06:59.190648 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.190517    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9" event={"ID":"28c5afbd-4c59-40ce-bcd2-ac4b708705ce","Type":"ContainerDied","Data":"52bd8e3d8936765a7a824357c0b8a9d33b1e889952d3ffc4d735d3dfa1440257"}
Sep 11 12:06:59.190648 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.190537    2586 scope.go:117] "RemoveContainer" containerID="a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822"
Sep 11 12:06:59.192545 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.192521    2586 generic.go:356] "Generic (PLEG): container finished" podID="268a16cc-a264-4779-aa0e-f5e8e34162ed" containerID="74b3642e2cdbcfd9ecb04188f92162b9bffc9e0a30dba15475f8cf2fe304dfe1" exitCode=0
Sep 11 12:06:59.192729 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.192606    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v" event={"ID":"268a16cc-a264-4779-aa0e-f5e8e34162ed","Type":"ContainerDied","Data":"74b3642e2cdbcfd9ecb04188f92162b9bffc9e0a30dba15475f8cf2fe304dfe1"}
Sep 11 12:06:59.192729 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.192695    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"
Sep 11 12:06:59.199261 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.199243    2586 scope.go:117] "RemoveContainer" containerID="a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822"
Sep 11 12:06:59.199576 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:06:59.199554    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822\": container with ID starting with a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822 not found: ID does not exist" containerID="a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822"
Sep 11 12:06:59.199657 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.199587    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822"} err="failed to get container status \"a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822\": rpc error: code = NotFound desc = could not find container \"a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822\": container with ID starting with a03d2a8d0f1f7149358b5250ce5a9fe9b284465a53f0024cafc14f4ed3c8e822 not found: ID does not exist"
Sep 11 12:06:59.209303 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.209282    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"]
Sep 11 12:06:59.212916 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.212896    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-d2d09be7-kserve-f99554b68-k9nq9"]
Sep 11 12:06:59.224197 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.224174    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"]
Sep 11 12:06:59.230226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.230207    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-d2d09be7-kserve-6f4fbd9f58-jtgz2"]
Sep 11 12:06:59.295620 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.295599    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:06:59.389389 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389350    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lffr\" (UniqueName: \"kubernetes.io/projected/268a16cc-a264-4779-aa0e-f5e8e34162ed-kube-api-access-9lffr\") pod \"268a16cc-a264-4779-aa0e-f5e8e34162ed\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") "
Sep 11 12:06:59.389569 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389398    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268a16cc-a264-4779-aa0e-f5e8e34162ed-tls-certs\") pod \"268a16cc-a264-4779-aa0e-f5e8e34162ed\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") "
Sep 11 12:06:59.389569 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389541    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-model-cache\") pod \"268a16cc-a264-4779-aa0e-f5e8e34162ed\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") "
Sep 11 12:06:59.389706 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389587    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-tmp-dir\") pod \"268a16cc-a264-4779-aa0e-f5e8e34162ed\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") "
Sep 11 12:06:59.389706 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389619    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-dshm\") pod \"268a16cc-a264-4779-aa0e-f5e8e34162ed\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") "
Sep 11 12:06:59.389706 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389643    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-home\") pod \"268a16cc-a264-4779-aa0e-f5e8e34162ed\" (UID: \"268a16cc-a264-4779-aa0e-f5e8e34162ed\") "
Sep 11 12:06:59.389876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389824    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "268a16cc-a264-4779-aa0e-f5e8e34162ed" (UID: "268a16cc-a264-4779-aa0e-f5e8e34162ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:59.389876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389839    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "268a16cc-a264-4779-aa0e-f5e8e34162ed" (UID: "268a16cc-a264-4779-aa0e-f5e8e34162ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:59.389988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389913    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-home" (OuterVolumeSpecName: "home") pod "268a16cc-a264-4779-aa0e-f5e8e34162ed" (UID: "268a16cc-a264-4779-aa0e-f5e8e34162ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:59.389988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389975    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:59.389988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.389988    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:59.391753 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.391728    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-dshm" (OuterVolumeSpecName: "dshm") pod "268a16cc-a264-4779-aa0e-f5e8e34162ed" (UID: "268a16cc-a264-4779-aa0e-f5e8e34162ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:06:59.391850 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.391771    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/268a16cc-a264-4779-aa0e-f5e8e34162ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "268a16cc-a264-4779-aa0e-f5e8e34162ed" (UID: "268a16cc-a264-4779-aa0e-f5e8e34162ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:59.391850 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.391837    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/268a16cc-a264-4779-aa0e-f5e8e34162ed-kube-api-access-9lffr" (OuterVolumeSpecName: "kube-api-access-9lffr") pod "268a16cc-a264-4779-aa0e-f5e8e34162ed" (UID: "268a16cc-a264-4779-aa0e-f5e8e34162ed"). InnerVolumeSpecName "kube-api-access-9lffr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:59.490365 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.490314    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:59.490365 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.490334    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268a16cc-a264-4779-aa0e-f5e8e34162ed-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:59.490365 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.490342    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lffr\" (UniqueName: \"kubernetes.io/projected/268a16cc-a264-4779-aa0e-f5e8e34162ed-kube-api-access-9lffr\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:06:59.490365 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:06:59.490353    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268a16cc-a264-4779-aa0e-f5e8e34162ed-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:07:00.199969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.199939    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"
Sep 11 12:07:00.199969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.199952    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v" event={"ID":"268a16cc-a264-4779-aa0e-f5e8e34162ed","Type":"ContainerDied","Data":"4eda48fbafc03b0ed2f0c809aa91942b47c3b16fb85b903299cca0b2c7c06b37"}
Sep 11 12:07:00.200455 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.200003    2586 scope.go:117] "RemoveContainer" containerID="74b3642e2cdbcfd9ecb04188f92162b9bffc9e0a30dba15475f8cf2fe304dfe1"
Sep 11 12:07:00.221965 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.221940    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"]
Sep 11 12:07:00.225811 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.225787    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-2c2847/autores-model-2c2847-kserve-67f474b6c5-n4j8v"]
Sep 11 12:07:00.824597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.824562    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="268a16cc-a264-4779-aa0e-f5e8e34162ed" path="/var/lib/kubelet/pods/268a16cc-a264-4779-aa0e-f5e8e34162ed/volumes"
Sep 11 12:07:00.824956 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.824942    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c5afbd-4c59-40ce-bcd2-ac4b708705ce" path="/var/lib/kubelet/pods/28c5afbd-4c59-40ce-bcd2-ac4b708705ce/volumes"
Sep 11 12:07:00.825275 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:00.825263    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b0df008-481f-487d-bf20-e2cddead6cf3" path="/var/lib/kubelet/pods/2b0df008-481f-487d-bf20-e2cddead6cf3/volumes"
Sep 11 12:07:01.117635 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:01.117547    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:01.221875 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:01.221813    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:01.322082 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:01.322056    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:01.422218 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:01.422135    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:01.484341 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:01.484302    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:01.879061 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:01.879036    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:02.195088 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:02.195003    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:02.308983 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:02.308943    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:04.803832 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:04.803797    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:04.902586 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:04.902554    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:06.029239 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.029212    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:06.125072 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.125039    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:06.500826 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.500793    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:06.605008 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.604978    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"]
Sep 11 12:07:06.605334 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605322    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="268a16cc-a264-4779-aa0e-f5e8e34162ed" containerName="main"
Sep 11 12:07:06.605392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605335    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="268a16cc-a264-4779-aa0e-f5e8e34162ed" containerName="main"
Sep 11 12:07:06.605392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605348    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2042335e-75f2-4cc6-a724-9044124748db" containerName="payload-processing"
Sep 11 12:07:06.605392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605363    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2042335e-75f2-4cc6-a724-9044124748db" containerName="payload-processing"
Sep 11 12:07:06.605392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605380    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5afbd-4c59-40ce-bcd2-ac4b708705ce" containerName="main"
Sep 11 12:07:06.605392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605386    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5afbd-4c59-40ce-bcd2-ac4b708705ce" containerName="main"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605397    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b0df008-481f-487d-bf20-e2cddead6cf3" containerName="main"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605402    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b0df008-481f-487d-bf20-e2cddead6cf3" containerName="main"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605409    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ee7460c-bced-4438-a1e9-1f107317ed52" containerName="payload-processing"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605417    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ee7460c-bced-4438-a1e9-1f107317ed52" containerName="payload-processing"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605476    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b0df008-481f-487d-bf20-e2cddead6cf3" containerName="main"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605502    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ee7460c-bced-4438-a1e9-1f107317ed52" containerName="payload-processing"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605512    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c5afbd-4c59-40ce-bcd2-ac4b708705ce" containerName="main"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605520    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="268a16cc-a264-4779-aa0e-f5e8e34162ed" containerName="main"
Sep 11 12:07:06.605615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.605526    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2042335e-75f2-4cc6-a724-9044124748db" containerName="payload-processing"
Sep 11 12:07:06.610279 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.610261    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.613526 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.613507    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-176ff0\""
Sep 11 12:07:06.620154 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.620132    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"]
Sep 11 12:07:06.629071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.629047    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"]
Sep 11 12:07:06.640343 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.640260    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.643738 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.643718    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-176ff0-dockercfg-tttjp\""
Sep 11 12:07:06.646089 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.646068    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"]
Sep 11 12:07:06.749309 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.749284    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gsvh\" (UniqueName: \"kubernetes.io/projected/750370cf-5171-47a2-ac29-2621332cd6e3-kube-api-access-5gsvh\") pod \"payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") " pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.749504 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.749330    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.749504 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.749365    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khj7v\" (UniqueName: \"kubernetes.io/projected/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-kube-api-access-khj7v\") pod \"payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.749504 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.749476    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750370cf-5171-47a2-ac29-2621332cd6e3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") " pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.850873 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.850842    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.851039 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.850877    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khj7v\" (UniqueName: \"kubernetes.io/projected/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-kube-api-access-khj7v\") pod \"payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.851039 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.850935    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750370cf-5171-47a2-ac29-2621332cd6e3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") " pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.851039 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.850971    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gsvh\" (UniqueName: \"kubernetes.io/projected/750370cf-5171-47a2-ac29-2621332cd6e3-kube-api-access-5gsvh\") pod \"payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") " pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.851834 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.851809    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.851918 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.851809    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750370cf-5171-47a2-ac29-2621332cd6e3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") " pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.862948 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.862929    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gsvh\" (UniqueName: \"kubernetes.io/projected/750370cf-5171-47a2-ac29-2621332cd6e3-kube-api-access-5gsvh\") pod \"payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") " pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.863075 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.862952    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khj7v\" (UniqueName: \"kubernetes.io/projected/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-kube-api-access-khj7v\") pod \"payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.886702 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.886673    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:06.921313 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.921288    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:06.951996 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.951968    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:06.993307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:06.993259    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:07.196843 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:07.196721    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"]
Sep 11 12:07:07.199590 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:07:07.199562    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6b553e_95fe_461b_9fa7_e6a1faedcacc.slice/crio-7cacea7329dce4e566d328eeac14e81fc4a8f544af4809a960121745c4b4d15b WatchSource:0}: Error finding container 7cacea7329dce4e566d328eeac14e81fc4a8f544af4809a960121745c4b4d15b: Status 404 returned error can't find the container with id 7cacea7329dce4e566d328eeac14e81fc4a8f544af4809a960121745c4b4d15b
Sep 11 12:07:07.238421 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:07.238394    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"]
Sep 11 12:07:07.239330 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:07.239306    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp" event={"ID":"5d6b553e-95fe-461b-9fa7-e6a1faedcacc","Type":"ContainerStarted","Data":"7cacea7329dce4e566d328eeac14e81fc4a8f544af4809a960121745c4b4d15b"}
Sep 11 12:07:07.239864 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:07:07.239829    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750370cf_5171_47a2_ac29_2621332cd6e3.slice/crio-3dcb3ed7562843117872344a5003eb1475e04c279b466fa8e7b4a02b13199267 WatchSource:0}: Error finding container 3dcb3ed7562843117872344a5003eb1475e04c279b466fa8e7b4a02b13199267: Status 404 returned error can't find the container with id 3dcb3ed7562843117872344a5003eb1475e04c279b466fa8e7b4a02b13199267
Sep 11 12:07:07.743898 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:07.743864    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:08.246858 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.246813    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s" event={"ID":"750370cf-5171-47a2-ac29-2621332cd6e3","Type":"ContainerStarted","Data":"ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f"}
Sep 11 12:07:08.247319 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.246886    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s" event={"ID":"750370cf-5171-47a2-ac29-2621332cd6e3","Type":"ContainerStarted","Data":"3dcb3ed7562843117872344a5003eb1475e04c279b466fa8e7b4a02b13199267"}
Sep 11 12:07:08.247319 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.246915    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:08.248661 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.248630    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp" event={"ID":"5d6b553e-95fe-461b-9fa7-e6a1faedcacc","Type":"ContainerStarted","Data":"1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec"}
Sep 11 12:07:08.248776 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.248694    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:08.267397 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.267348    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s" podStartSLOduration=2.267333203 podStartE2EDuration="2.267333203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:08.265611253 +0000 UTC m=+921.976282587" watchObservedRunningTime="2026-09-11 12:07:08.267333203 +0000 UTC m=+921.978004541"
Sep 11 12:07:08.296224 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.296179    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp" podStartSLOduration=2.296166969 podStartE2EDuration="2.296166969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:08.294058864 +0000 UTC m=+922.004730201" watchObservedRunningTime="2026-09-11 12:07:08.296166969 +0000 UTC m=+922.006838307"
Sep 11 12:07:08.908333 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.908305    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:08.967551 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:08.967519    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:14.200275 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:14.200239    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:07:19.255024 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:19.254993    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:07:19.255509 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:19.255071    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:07:21.122165 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.122130    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"]
Sep 11 12:07:21.125606 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.125585    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.130101 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.130077    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e42660\"/\"test-model-e42660-kserve-self-signed-certs\""
Sep 11 12:07:21.130221 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.130106    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e42660\"/\"openshift-service-ca.crt\""
Sep 11 12:07:21.130221 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.130130    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e42660\"/\"kube-root-ca.crt\""
Sep 11 12:07:21.130313 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.130235    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e42660\"/\"default-dockercfg-65mxv\""
Sep 11 12:07:21.135684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.135660    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"]
Sep 11 12:07:21.153043 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.153014    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:21.174462 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.174437    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7192c81a-2f04-48e2-b506-a8aa442efb3a-tls-certs\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.174590 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.174477    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-dshm\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.174590 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.174557    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-model-cache\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.174692 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.174599    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-home\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.174692 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.174618    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68cn9\" (UniqueName: \"kubernetes.io/projected/7192c81a-2f04-48e2-b506-a8aa442efb3a-kube-api-access-68cn9\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.174798 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.174727    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-tmp-dir\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.275770 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.275741    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7192c81a-2f04-48e2-b506-a8aa442efb3a-tls-certs\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.275907 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.275781    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-dshm\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.275907 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.275802    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-model-cache\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.275907 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.275839    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-home\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.275907 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.275857    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68cn9\" (UniqueName: \"kubernetes.io/projected/7192c81a-2f04-48e2-b506-a8aa442efb3a-kube-api-access-68cn9\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.275907 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.275886    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-tmp-dir\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.276259 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.276238    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-model-cache\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.276328 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.276279    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-home\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.276328 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.276313    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-tmp-dir\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.278108 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.278088    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-dshm\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.278446 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.278429    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7192c81a-2f04-48e2-b506-a8aa442efb3a-tls-certs\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.284736 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.284718    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68cn9\" (UniqueName: \"kubernetes.io/projected/7192c81a-2f04-48e2-b506-a8aa442efb3a-kube-api-access-68cn9\") pod \"test-model-e42660-kserve-68599d47ff-564kw\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") " pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.437209 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.437134    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:21.709732 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:21.709707    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"]
Sep 11 12:07:21.711529 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:07:21.711502    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7192c81a_2f04_48e2_b506_a8aa442efb3a.slice/crio-3252954f9f9a5ed96aeaae40d185815fa0fe873db537a645d0b888bdc9663217 WatchSource:0}: Error finding container 3252954f9f9a5ed96aeaae40d185815fa0fe873db537a645d0b888bdc9663217: Status 404 returned error can't find the container with id 3252954f9f9a5ed96aeaae40d185815fa0fe873db537a645d0b888bdc9663217
Sep 11 12:07:22.317876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:22.317837    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw" event={"ID":"7192c81a-2f04-48e2-b506-a8aa442efb3a","Type":"ContainerStarted","Data":"39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49"}
Sep 11 12:07:22.318242 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:22.317881    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw" event={"ID":"7192c81a-2f04-48e2-b506-a8aa442efb3a","Type":"ContainerStarted","Data":"3252954f9f9a5ed96aeaae40d185815fa0fe873db537a645d0b888bdc9663217"}
Sep 11 12:07:22.336373 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:22.336326    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw" podStartSLOduration=1.336310606 podStartE2EDuration="1.336310606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:22.334427025 +0000 UTC m=+936.045098365" watchObservedRunningTime="2026-09-11 12:07:22.336310606 +0000 UTC m=+936.046981944"
Sep 11 12:07:24.194075 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:24.194037    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:29.860324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:29.860288    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:30.193322 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:30.193238    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:30.317402 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:30.317367    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:30.466630 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:30.466552    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:31.437806 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:31.437775    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:31.438188 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:31.437847    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:31.450038 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:31.450015    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:32.375292 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:32.375264    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:07:36.303990 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:36.303956    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:07:40.841235 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:40.841203    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:41.561638 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:41.561602    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:47.533714 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:47.533675    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:48.718394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:48.718353    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:07:49.073169 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:49.073136    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:50.036410 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:50.036376    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:56.072706 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.072672    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:56.173197 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.173167    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:56.337898 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.337827    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:56.607090 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.607005    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"]
Sep 11 12:07:56.612133 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.612113    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.616629 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.616609    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-176ff0\"/\"default-dockercfg-br2nw\""
Sep 11 12:07:56.616760 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.616641    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-176ff0\"/\"explicit-model-176ff0-kserve-self-signed-certs\""
Sep 11 12:07:56.616760 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.616617    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-176ff0\"/\"openshift-service-ca.crt\""
Sep 11 12:07:56.616760 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.616716    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-176ff0\"/\"kube-root-ca.crt\""
Sep 11 12:07:56.619828 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.619804    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"]
Sep 11 12:07:56.641234 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.641213    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:07:56.700757 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.700727    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9g75\" (UniqueName: \"kubernetes.io/projected/b4e2d928-c029-4751-893b-39789584fbbc-kube-api-access-s9g75\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.700902 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.700769    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-dshm\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.700902 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.700845    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-model-cache\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.700902 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.700862    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e2d928-c029-4751-893b-39789584fbbc-tls-certs\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.701071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.700933    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-home\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.701071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.700959    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-tmp-dir\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802219 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802187    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-tmp-dir\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802230    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9g75\" (UniqueName: \"kubernetes.io/projected/b4e2d928-c029-4751-893b-39789584fbbc-kube-api-access-s9g75\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802253    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-dshm\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802296    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-model-cache\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802311    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e2d928-c029-4751-893b-39789584fbbc-tls-certs\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802344    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-home\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802687 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802665    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-tmp-dir\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802750 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802723    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-model-cache\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.802804 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.802746    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-home\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.804734 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.804713    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-dshm\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.804963 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.804946    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e2d928-c029-4751-893b-39789584fbbc-tls-certs\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.821112 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.821093    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9g75\" (UniqueName: \"kubernetes.io/projected/b4e2d928-c029-4751-893b-39789584fbbc-kube-api-access-s9g75\") pod \"explicit-model-176ff0-kserve-5c789546d9-qzcvm\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") " pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:56.922466 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:56.922407    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:07:57.206213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:57.206186    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"]
Sep 11 12:07:57.207968 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:07:57.207935    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4e2d928_c029_4751_893b_39789584fbbc.slice/crio-13fa126e631bef20a46cf1b2ed31946d50d0a7fc7a51f49a192f6c2020cae4ca WatchSource:0}: Error finding container 13fa126e631bef20a46cf1b2ed31946d50d0a7fc7a51f49a192f6c2020cae4ca: Status 404 returned error can't find the container with id 13fa126e631bef20a46cf1b2ed31946d50d0a7fc7a51f49a192f6c2020cae4ca
Sep 11 12:07:57.485886 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:57.485799    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm" event={"ID":"b4e2d928-c029-4751-893b-39789584fbbc","Type":"ContainerStarted","Data":"a6fe62aa325f9934080176b2cc341b2bcc1d2c958d6ac2c51a75403f771bd3f1"}
Sep 11 12:07:57.485886 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:57.485848    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm" event={"ID":"b4e2d928-c029-4751-893b-39789584fbbc","Type":"ContainerStarted","Data":"13fa126e631bef20a46cf1b2ed31946d50d0a7fc7a51f49a192f6c2020cae4ca"}
Sep 11 12:07:57.504955 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:07:57.504908    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm" podStartSLOduration=1.5048951879999999 podStartE2EDuration="1.504895188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:57.50304363 +0000 UTC m=+971.213714970" watchObservedRunningTime="2026-09-11 12:07:57.504895188 +0000 UTC m=+971.215566517"
Sep 11 12:08:02.291349 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:02.291316    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:02.359179 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:02.359152    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:02.526807 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:02.526781    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:02.830328 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:02.830298    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:03.004413 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:03.004382    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:04.709469 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:04.709434    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:08:06.217121 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:06.217082    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:06.880875 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:06.880846    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:06.923111 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:06.923090    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:08:06.923239 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:06.923126    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:08:06.935866 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:06.935846    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:08:07.364925 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:07.364888    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:07.470804 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:07.470773    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:07.548473 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:07.548445    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:08:08.452833 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:08.452738    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:08.528733 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:08.528697    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:21.440978 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:21.440949    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"]
Sep 11 12:08:21.465749 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:21.465717    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:21.678721 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:21.678684    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:24.401358 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:24.401309    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:08:29.484311 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:29.484277    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:31.154711 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:31.154673    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:31.692576 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:31.692540    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:31.794662 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:31.794631    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:32.558920 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.558882    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"]
Sep 11 12:08:32.559398 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.559109    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s" podUID="750370cf-5171-47a2-ac29-2621332cd6e3" containerName="payload-processing" containerID="cri-o://ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f" gracePeriod=30
Sep 11 12:08:32.573362 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.573324    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"]
Sep 11 12:08:32.573597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.573572    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp" podUID="5d6b553e-95fe-461b-9fa7-e6a1faedcacc" containerName="payload-pre-processing" containerID="cri-o://1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec" gracePeriod=30
Sep 11 12:08:32.829670 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.829600    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:08:32.854571 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.854546    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:08:32.897195 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.897167    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gsvh\" (UniqueName: \"kubernetes.io/projected/750370cf-5171-47a2-ac29-2621332cd6e3-kube-api-access-5gsvh\") pod \"750370cf-5171-47a2-ac29-2621332cd6e3\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") "
Sep 11 12:08:32.897195 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.897197    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khj7v\" (UniqueName: \"kubernetes.io/projected/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-kube-api-access-khj7v\") pod \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") "
Sep 11 12:08:32.897415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.897215    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750370cf-5171-47a2-ac29-2621332cd6e3-plugins-config-volume\") pod \"750370cf-5171-47a2-ac29-2621332cd6e3\" (UID: \"750370cf-5171-47a2-ac29-2621332cd6e3\") "
Sep 11 12:08:32.897415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.897248    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-plugins-config-volume\") pod \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\" (UID: \"5d6b553e-95fe-461b-9fa7-e6a1faedcacc\") "
Sep 11 12:08:32.897676 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.897653    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/750370cf-5171-47a2-ac29-2621332cd6e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "750370cf-5171-47a2-ac29-2621332cd6e3" (UID: "750370cf-5171-47a2-ac29-2621332cd6e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:32.897740 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.897667    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d6b553e-95fe-461b-9fa7-e6a1faedcacc" (UID: "5d6b553e-95fe-461b-9fa7-e6a1faedcacc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:32.899495 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.899461    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-kube-api-access-khj7v" (OuterVolumeSpecName: "kube-api-access-khj7v") pod "5d6b553e-95fe-461b-9fa7-e6a1faedcacc" (UID: "5d6b553e-95fe-461b-9fa7-e6a1faedcacc"). InnerVolumeSpecName "kube-api-access-khj7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:32.899635 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.899613    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/750370cf-5171-47a2-ac29-2621332cd6e3-kube-api-access-5gsvh" (OuterVolumeSpecName: "kube-api-access-5gsvh") pod "750370cf-5171-47a2-ac29-2621332cd6e3" (UID: "750370cf-5171-47a2-ac29-2621332cd6e3"). InnerVolumeSpecName "kube-api-access-5gsvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:32.998897 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.998874    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gsvh\" (UniqueName: \"kubernetes.io/projected/750370cf-5171-47a2-ac29-2621332cd6e3-kube-api-access-5gsvh\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:32.998897 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.998896    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khj7v\" (UniqueName: \"kubernetes.io/projected/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-kube-api-access-khj7v\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:32.999051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.998905    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750370cf-5171-47a2-ac29-2621332cd6e3-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:32.999051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:32.998914    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6b553e-95fe-461b-9fa7-e6a1faedcacc-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:33.666217 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.666186    2586 generic.go:356] "Generic (PLEG): container finished" podID="750370cf-5171-47a2-ac29-2621332cd6e3" containerID="ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f" exitCode=0
Sep 11 12:08:33.666640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.666232    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"
Sep 11 12:08:33.666640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.666282    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s" event={"ID":"750370cf-5171-47a2-ac29-2621332cd6e3","Type":"ContainerDied","Data":"ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f"}
Sep 11 12:08:33.666640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.666347    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s" event={"ID":"750370cf-5171-47a2-ac29-2621332cd6e3","Type":"ContainerDied","Data":"3dcb3ed7562843117872344a5003eb1475e04c279b466fa8e7b4a02b13199267"}
Sep 11 12:08:33.666640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.666379    2586 scope.go:117] "RemoveContainer" containerID="ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f"
Sep 11 12:08:33.667816 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.667643    2586 generic.go:356] "Generic (PLEG): container finished" podID="5d6b553e-95fe-461b-9fa7-e6a1faedcacc" containerID="1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec" exitCode=0
Sep 11 12:08:33.667816 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.667685    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"
Sep 11 12:08:33.667816 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.667693    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp" event={"ID":"5d6b553e-95fe-461b-9fa7-e6a1faedcacc","Type":"ContainerDied","Data":"1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec"}
Sep 11 12:08:33.667816 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.667745    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp" event={"ID":"5d6b553e-95fe-461b-9fa7-e6a1faedcacc","Type":"ContainerDied","Data":"7cacea7329dce4e566d328eeac14e81fc4a8f544af4809a960121745c4b4d15b"}
Sep 11 12:08:33.676824 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.676808    2586 scope.go:117] "RemoveContainer" containerID="ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f"
Sep 11 12:08:33.677098 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:08:33.677076    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f\": container with ID starting with ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f not found: ID does not exist" containerID="ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f"
Sep 11 12:08:33.677168 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.677106    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f"} err="failed to get container status \"ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f\": rpc error: code = NotFound desc = could not find container \"ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f\": container with ID starting with ebd24d4a76c21819bffc419a4aa7708c05921008298cb7b00eebc30074870e8f not found: ID does not exist"
Sep 11 12:08:33.677168 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.677122    2586 scope.go:117] "RemoveContainer" containerID="1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec"
Sep 11 12:08:33.685067 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.685051    2586 scope.go:117] "RemoveContainer" containerID="1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec"
Sep 11 12:08:33.685324 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:08:33.685307    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec\": container with ID starting with 1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec not found: ID does not exist" containerID="1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec"
Sep 11 12:08:33.685384 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.685332    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec"} err="failed to get container status \"1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec\": rpc error: code = NotFound desc = could not find container \"1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec\": container with ID starting with 1852e9c51c42b33976513b1e201733259f45005d49f4979d4423538bacffe7ec not found: ID does not exist"
Sep 11 12:08:33.692784 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.692765    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"]
Sep 11 12:08:33.697250 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.697230    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-176ff0-7bd4cdfc49-svx2s"]
Sep 11 12:08:33.709804 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.709787    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"]
Sep 11 12:08:33.713888 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.713867    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-176ff0-8d6986f9d-thdmp"]
Sep 11 12:08:33.740018 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.739980    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:33.806007 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:33.805979    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:34.823979 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:34.823946    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d6b553e-95fe-461b-9fa7-e6a1faedcacc" path="/var/lib/kubelet/pods/5d6b553e-95fe-461b-9fa7-e6a1faedcacc/volumes"
Sep 11 12:08:34.824358 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:34.824286    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="750370cf-5171-47a2-ac29-2621332cd6e3" path="/var/lib/kubelet/pods/750370cf-5171-47a2-ac29-2621332cd6e3/volumes"
Sep 11 12:08:36.465625 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.465589    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm" podUID="b4e2d928-c029-4751-893b-39789584fbbc" containerName="main" containerID="cri-o://a6fe62aa325f9934080176b2cc341b2bcc1d2c958d6ac2c51a75403f771bd3f1" gracePeriod=45
Sep 11 12:08:36.555232 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.555205    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:36.686450 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.686423    2586 generic.go:356] "Generic (PLEG): container finished" podID="b4e2d928-c029-4751-893b-39789584fbbc" containerID="a6fe62aa325f9934080176b2cc341b2bcc1d2c958d6ac2c51a75403f771bd3f1" exitCode=0
Sep 11 12:08:36.686615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.686473    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm" event={"ID":"b4e2d928-c029-4751-893b-39789584fbbc","Type":"ContainerDied","Data":"a6fe62aa325f9934080176b2cc341b2bcc1d2c958d6ac2c51a75403f771bd3f1"}
Sep 11 12:08:36.719400 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.719349    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:08:36.826085 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826063    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-tmp-dir\") pod \"b4e2d928-c029-4751-893b-39789584fbbc\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") "
Sep 11 12:08:36.826227 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826105    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e2d928-c029-4751-893b-39789584fbbc-tls-certs\") pod \"b4e2d928-c029-4751-893b-39789584fbbc\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") "
Sep 11 12:08:36.826227 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826124    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9g75\" (UniqueName: \"kubernetes.io/projected/b4e2d928-c029-4751-893b-39789584fbbc-kube-api-access-s9g75\") pod \"b4e2d928-c029-4751-893b-39789584fbbc\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") "
Sep 11 12:08:36.826227 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826145    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-dshm\") pod \"b4e2d928-c029-4751-893b-39789584fbbc\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") "
Sep 11 12:08:36.826227 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826183    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-home\") pod \"b4e2d928-c029-4751-893b-39789584fbbc\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") "
Sep 11 12:08:36.826524 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826243    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-model-cache\") pod \"b4e2d928-c029-4751-893b-39789584fbbc\" (UID: \"b4e2d928-c029-4751-893b-39789584fbbc\") "
Sep 11 12:08:36.826524 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826300    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4e2d928-c029-4751-893b-39789584fbbc" (UID: "b4e2d928-c029-4751-893b-39789584fbbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:36.826627 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826538    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-home" (OuterVolumeSpecName: "home") pod "b4e2d928-c029-4751-893b-39789584fbbc" (UID: "b4e2d928-c029-4751-893b-39789584fbbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:36.826627 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826548    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4e2d928-c029-4751-893b-39789584fbbc" (UID: "b4e2d928-c029-4751-893b-39789584fbbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:36.826627 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.826557    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:36.828405 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.828374    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-dshm" (OuterVolumeSpecName: "dshm") pod "b4e2d928-c029-4751-893b-39789584fbbc" (UID: "b4e2d928-c029-4751-893b-39789584fbbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:36.828531 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.828445    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4e2d928-c029-4751-893b-39789584fbbc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4e2d928-c029-4751-893b-39789584fbbc" (UID: "b4e2d928-c029-4751-893b-39789584fbbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:36.828531 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.828459    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e2d928-c029-4751-893b-39789584fbbc-kube-api-access-s9g75" (OuterVolumeSpecName: "kube-api-access-s9g75") pod "b4e2d928-c029-4751-893b-39789584fbbc" (UID: "b4e2d928-c029-4751-893b-39789584fbbc"). InnerVolumeSpecName "kube-api-access-s9g75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:36.927339 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.927316    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e2d928-c029-4751-893b-39789584fbbc-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:36.927339 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.927337    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9g75\" (UniqueName: \"kubernetes.io/projected/b4e2d928-c029-4751-893b-39789584fbbc-kube-api-access-s9g75\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:36.927468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.927347    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:36.927468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.927355    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:36.927468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:36.927363    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e2d928-c029-4751-893b-39789584fbbc-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:37.692788 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:37.692749    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm" event={"ID":"b4e2d928-c029-4751-893b-39789584fbbc","Type":"ContainerDied","Data":"13fa126e631bef20a46cf1b2ed31946d50d0a7fc7a51f49a192f6c2020cae4ca"}
Sep 11 12:08:37.692788 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:37.692790    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"
Sep 11 12:08:37.693287 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:37.692812    2586 scope.go:117] "RemoveContainer" containerID="a6fe62aa325f9934080176b2cc341b2bcc1d2c958d6ac2c51a75403f771bd3f1"
Sep 11 12:08:37.714147 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:37.714120    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"]
Sep 11 12:08:37.718595 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:37.718573    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-176ff0/explicit-model-176ff0-kserve-5c789546d9-qzcvm"]
Sep 11 12:08:38.157147 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:38.157119    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:38.259832 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:38.259804    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:38.309399 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:38.309373    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:08:38.544296 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:38.544267    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:38.682359 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:38.682329    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:38.823256 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:38.823194    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e2d928-c029-4751-893b-39789584fbbc" path="/var/lib/kubelet/pods/b4e2d928-c029-4751-893b-39789584fbbc/volumes"
Sep 11 12:08:40.086004 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.085974    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"]
Sep 11 12:08:40.109510 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.109457    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:40.200411 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200384    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"]
Sep 11 12:08:40.200807 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200793    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="750370cf-5171-47a2-ac29-2621332cd6e3" containerName="payload-processing"
Sep 11 12:08:40.200861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200810    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="750370cf-5171-47a2-ac29-2621332cd6e3" containerName="payload-processing"
Sep 11 12:08:40.200861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200822    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d6b553e-95fe-461b-9fa7-e6a1faedcacc" containerName="payload-pre-processing"
Sep 11 12:08:40.200861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200827    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d6b553e-95fe-461b-9fa7-e6a1faedcacc" containerName="payload-pre-processing"
Sep 11 12:08:40.200861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200837    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e2d928-c029-4751-893b-39789584fbbc" containerName="main"
Sep 11 12:08:40.200861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200843    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e2d928-c029-4751-893b-39789584fbbc" containerName="main"
Sep 11 12:08:40.201072 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200914    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="750370cf-5171-47a2-ac29-2621332cd6e3" containerName="payload-processing"
Sep 11 12:08:40.201072 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200926    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e2d928-c029-4751-893b-39789584fbbc" containerName="main"
Sep 11 12:08:40.201072 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.200932    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d6b553e-95fe-461b-9fa7-e6a1faedcacc" containerName="payload-pre-processing"
Sep 11 12:08:40.205230 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.205215    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"
Sep 11 12:08:40.206812 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.206784    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:40.208138 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.208118    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:08:40.208755 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.208741    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:08:40.213533 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.213513    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"]
Sep 11 12:08:40.308314 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.308281    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:40.356252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.356191    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fba1a6-abf3-4002-818b-eed4edceb9fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx\" (UID: \"82fba1a6-abf3-4002-818b-eed4edceb9fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"
Sep 11 12:08:40.445745 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.445718    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:40.457773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.457741    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fba1a6-abf3-4002-818b-eed4edceb9fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx\" (UID: \"82fba1a6-abf3-4002-818b-eed4edceb9fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"
Sep 11 12:08:40.458316 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.458298    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fba1a6-abf3-4002-818b-eed4edceb9fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx\" (UID: \"82fba1a6-abf3-4002-818b-eed4edceb9fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"
Sep 11 12:08:40.516713 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.516689    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"
Sep 11 12:08:40.545565 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.545539    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:40.788983 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.788961    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"]
Sep 11 12:08:40.792148 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:08:40.792118    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fba1a6_abf3_4002_818b_eed4edceb9fd.slice/crio-c19009cc95deff04a029a2e90a84a9d49395e34ea5e53572b2ebaf6b6c165068 WatchSource:0}: Error finding container c19009cc95deff04a029a2e90a84a9d49395e34ea5e53572b2ebaf6b6c165068: Status 404 returned error can't find the container with id c19009cc95deff04a029a2e90a84a9d49395e34ea5e53572b2ebaf6b6c165068
Sep 11 12:08:40.794133 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.794115    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:08:40.905997 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:40.905970    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:41.151096 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:41.150990    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:41.294250 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:41.294219    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:41.717597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:41.717554    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx" event={"ID":"82fba1a6-abf3-4002-818b-eed4edceb9fd","Type":"ContainerStarted","Data":"c19009cc95deff04a029a2e90a84a9d49395e34ea5e53572b2ebaf6b6c165068"}
Sep 11 12:08:42.885086 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:42.885047    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:42.984316 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:42.984285    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:43.103851 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:43.103796    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:44.736137 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:44.736069    2586 generic.go:356] "Generic (PLEG): container finished" podID="82fba1a6-abf3-4002-818b-eed4edceb9fd" containerID="0cc55a08a6f84dd7d9c3bde8379a0cf0dc2eb58b00e1fdc14b6592c876b96fcd" exitCode=0
Sep 11 12:08:44.736468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:44.736160    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx" event={"ID":"82fba1a6-abf3-4002-818b-eed4edceb9fd","Type":"ContainerDied","Data":"0cc55a08a6f84dd7d9c3bde8379a0cf0dc2eb58b00e1fdc14b6592c876b96fcd"}
Sep 11 12:08:45.878786 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:45.878761    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"
Sep 11 12:08:46.006023 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:46.005965    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fba1a6-abf3-4002-818b-eed4edceb9fd-maas-api-service-ca\") pod \"82fba1a6-abf3-4002-818b-eed4edceb9fd\" (UID: \"82fba1a6-abf3-4002-818b-eed4edceb9fd\") "
Sep 11 12:08:46.006277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:46.006255    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82fba1a6-abf3-4002-818b-eed4edceb9fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82fba1a6-abf3-4002-818b-eed4edceb9fd" (UID: "82fba1a6-abf3-4002-818b-eed4edceb9fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:46.107476 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:46.107452    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fba1a6-abf3-4002-818b-eed4edceb9fd-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:46.748695 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:46.748666    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx" event={"ID":"82fba1a6-abf3-4002-818b-eed4edceb9fd","Type":"ContainerDied","Data":"c19009cc95deff04a029a2e90a84a9d49395e34ea5e53572b2ebaf6b6c165068"}
Sep 11 12:08:46.748695 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:46.748698    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"
Sep 11 12:08:46.748901 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:46.748743    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c19009cc95deff04a029a2e90a84a9d49395e34ea5e53572b2ebaf6b6c165068"
Sep 11 12:08:48.666473 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:48.666433    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:49.363360 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:49.363324    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:49.463611 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:49.463583    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:49.563425 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:49.563392    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:49.628383 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:49.628310    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:49.969340 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:49.969237    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:50.371286 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:50.371256    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:51.101410 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.101378    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:51.134001 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.133976    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"]
Sep 11 12:08:51.134230 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.134205    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8" podUID="b2b9a1a6-7d81-41cf-89b2-1355778569ae" containerName="payload-processing" containerID="cri-o://77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e" gracePeriod=30
Sep 11 12:08:51.202282 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.202254    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:51.302296 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.302262    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:51.319539 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.319507    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"]
Sep 11 12:08:51.323254 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.323231    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-34511f-9jrjx"]
Sep 11 12:08:51.374997 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.374971    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:51.392543 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.392523    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:08:51.553716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.553686    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdv5g\" (UniqueName: \"kubernetes.io/projected/b2b9a1a6-7d81-41cf-89b2-1355778569ae-kube-api-access-cdv5g\") pod \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") "
Sep 11 12:08:51.553878 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.553779    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b9a1a6-7d81-41cf-89b2-1355778569ae-plugins-config-volume\") pod \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\" (UID: \"b2b9a1a6-7d81-41cf-89b2-1355778569ae\") "
Sep 11 12:08:51.554168 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.554142    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2b9a1a6-7d81-41cf-89b2-1355778569ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2b9a1a6-7d81-41cf-89b2-1355778569ae" (UID: "b2b9a1a6-7d81-41cf-89b2-1355778569ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:51.555902 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.555875    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b9a1a6-7d81-41cf-89b2-1355778569ae-kube-api-access-cdv5g" (OuterVolumeSpecName: "kube-api-access-cdv5g") pod "b2b9a1a6-7d81-41cf-89b2-1355778569ae" (UID: "b2b9a1a6-7d81-41cf-89b2-1355778569ae"). InnerVolumeSpecName "kube-api-access-cdv5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:51.654931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.654871    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdv5g\" (UniqueName: \"kubernetes.io/projected/b2b9a1a6-7d81-41cf-89b2-1355778569ae-kube-api-access-cdv5g\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:51.654931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.654895    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b9a1a6-7d81-41cf-89b2-1355778569ae-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:51.672527 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.672474    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:51.775778 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.775751    2586 generic.go:356] "Generic (PLEG): container finished" podID="b2b9a1a6-7d81-41cf-89b2-1355778569ae" containerID="77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e" exitCode=0
Sep 11 12:08:51.775893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.775782    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8" event={"ID":"b2b9a1a6-7d81-41cf-89b2-1355778569ae","Type":"ContainerDied","Data":"77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e"}
Sep 11 12:08:51.775893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.775811    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"
Sep 11 12:08:51.775893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.775852    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8" event={"ID":"b2b9a1a6-7d81-41cf-89b2-1355778569ae","Type":"ContainerDied","Data":"1d1ded077b372bd34794b52c6d76b36ccdaa6b754266fc682dc8a7b0f25765df"}
Sep 11 12:08:51.775893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.775873    2586 scope.go:117] "RemoveContainer" containerID="77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e"
Sep 11 12:08:51.786468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.786440    2586 scope.go:117] "RemoveContainer" containerID="77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e"
Sep 11 12:08:51.786862 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:08:51.786838    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e\": container with ID starting with 77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e not found: ID does not exist" containerID="77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e"
Sep 11 12:08:51.786962 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.786875    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e"} err="failed to get container status \"77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e\": rpc error: code = NotFound desc = could not find container \"77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e\": container with ID starting with 77a4e32d1990094a8574d9833e442880e146a899abaf1f7a528270dace6e5f0e not found: ID does not exist"
Sep 11 12:08:51.800750 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.800729    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"]
Sep 11 12:08:51.804586 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:51.804567    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34511f-84d5646c77-rhwv8"]
Sep 11 12:08:52.103208 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:52.103181    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:08:52.823725 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:52.823692    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82fba1a6-abf3-4002-818b-eed4edceb9fd" path="/var/lib/kubelet/pods/82fba1a6-abf3-4002-818b-eed4edceb9fd/volumes"
Sep 11 12:08:52.824032 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:52.824019    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2b9a1a6-7d81-41cf-89b2-1355778569ae" path="/var/lib/kubelet/pods/b2b9a1a6-7d81-41cf-89b2-1355778569ae/volumes"
Sep 11 12:08:55.105006 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.104971    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw" podUID="7192c81a-2f04-48e2-b506-a8aa442efb3a" containerName="main" containerID="cri-o://39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49" gracePeriod=45
Sep 11 12:08:55.251102 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.251072    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:55.353852 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.353828    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:08:55.483019 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.482962    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-tmp-dir\") pod \"7192c81a-2f04-48e2-b506-a8aa442efb3a\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") "
Sep 11 12:08:55.483019 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.482999    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7192c81a-2f04-48e2-b506-a8aa442efb3a-tls-certs\") pod \"7192c81a-2f04-48e2-b506-a8aa442efb3a\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") "
Sep 11 12:08:55.483213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483032    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-model-cache\") pod \"7192c81a-2f04-48e2-b506-a8aa442efb3a\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") "
Sep 11 12:08:55.483213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483051    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-dshm\") pod \"7192c81a-2f04-48e2-b506-a8aa442efb3a\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") "
Sep 11 12:08:55.483213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483098    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68cn9\" (UniqueName: \"kubernetes.io/projected/7192c81a-2f04-48e2-b506-a8aa442efb3a-kube-api-access-68cn9\") pod \"7192c81a-2f04-48e2-b506-a8aa442efb3a\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") "
Sep 11 12:08:55.483213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483174    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-home\") pod \"7192c81a-2f04-48e2-b506-a8aa442efb3a\" (UID: \"7192c81a-2f04-48e2-b506-a8aa442efb3a\") "
Sep 11 12:08:55.483213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483179    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7192c81a-2f04-48e2-b506-a8aa442efb3a" (UID: "7192c81a-2f04-48e2-b506-a8aa442efb3a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:55.483446 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483210    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-model-cache" (OuterVolumeSpecName: "model-cache") pod "7192c81a-2f04-48e2-b506-a8aa442efb3a" (UID: "7192c81a-2f04-48e2-b506-a8aa442efb3a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:55.483538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483462    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:55.483538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483462    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-home" (OuterVolumeSpecName: "home") pod "7192c81a-2f04-48e2-b506-a8aa442efb3a" (UID: "7192c81a-2f04-48e2-b506-a8aa442efb3a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:55.483538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.483507    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:55.485388 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.485366    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7192c81a-2f04-48e2-b506-a8aa442efb3a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7192c81a-2f04-48e2-b506-a8aa442efb3a" (UID: "7192c81a-2f04-48e2-b506-a8aa442efb3a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:55.485473 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.485397    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-dshm" (OuterVolumeSpecName: "dshm") pod "7192c81a-2f04-48e2-b506-a8aa442efb3a" (UID: "7192c81a-2f04-48e2-b506-a8aa442efb3a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:55.485473 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.485466    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7192c81a-2f04-48e2-b506-a8aa442efb3a-kube-api-access-68cn9" (OuterVolumeSpecName: "kube-api-access-68cn9") pod "7192c81a-2f04-48e2-b506-a8aa442efb3a" (UID: "7192c81a-2f04-48e2-b506-a8aa442efb3a"). InnerVolumeSpecName "kube-api-access-68cn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:55.584097 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.584060    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:55.584097 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.584093    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68cn9\" (UniqueName: \"kubernetes.io/projected/7192c81a-2f04-48e2-b506-a8aa442efb3a-kube-api-access-68cn9\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:55.584097 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.584103    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7192c81a-2f04-48e2-b506-a8aa442efb3a-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:55.584097 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.584112    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7192c81a-2f04-48e2-b506-a8aa442efb3a-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:08:55.799839 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.799811    2586 generic.go:356] "Generic (PLEG): container finished" podID="7192c81a-2f04-48e2-b506-a8aa442efb3a" containerID="39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49" exitCode=0
Sep 11 12:08:55.799964 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.799859    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"
Sep 11 12:08:55.799964 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.799878    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw" event={"ID":"7192c81a-2f04-48e2-b506-a8aa442efb3a","Type":"ContainerDied","Data":"39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49"}
Sep 11 12:08:55.799964 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.799924    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw" event={"ID":"7192c81a-2f04-48e2-b506-a8aa442efb3a","Type":"ContainerDied","Data":"3252954f9f9a5ed96aeaae40d185815fa0fe873db537a645d0b888bdc9663217"}
Sep 11 12:08:55.799964 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.799945    2586 scope.go:117] "RemoveContainer" containerID="39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49"
Sep 11 12:08:55.809106 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.809092    2586 scope.go:117] "RemoveContainer" containerID="39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49"
Sep 11 12:08:55.809396 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:08:55.809374    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49\": container with ID starting with 39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49 not found: ID does not exist" containerID="39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49"
Sep 11 12:08:55.809441 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.809403    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49"} err="failed to get container status \"39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49\": rpc error: code = NotFound desc = could not find container \"39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49\": container with ID starting with 39eacb38b8e846a3b534e009b3ad25e4150e8e691a7aaa395ecb3d227e361b49 not found: ID does not exist"
Sep 11 12:08:55.824664 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.824642    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"]
Sep 11 12:08:55.829045 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:55.829024    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e42660/test-model-e42660-kserve-68599d47ff-564kw"]
Sep 11 12:08:56.471597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:56.471558    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:56.548348 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:56.548325    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:56.740518 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:56.740391    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:56.822854 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:56.822828    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7192c81a-2f04-48e2-b506-a8aa442efb3a" path="/var/lib/kubelet/pods/7192c81a-2f04-48e2-b506-a8aa442efb3a/volumes"
Sep 11 12:08:57.094049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:57.094019    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:57.185085 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:57.185056    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:57.295227 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:57.295200    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:58.136182 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.136147    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:58.236251 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.236216    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:58.378220 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.378173    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:58.692350 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692320    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"]
Sep 11 12:08:58.692675 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692663    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2b9a1a6-7d81-41cf-89b2-1355778569ae" containerName="payload-processing"
Sep 11 12:08:58.692728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692677    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2b9a1a6-7d81-41cf-89b2-1355778569ae" containerName="payload-processing"
Sep 11 12:08:58.692728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692687    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7192c81a-2f04-48e2-b506-a8aa442efb3a" containerName="main"
Sep 11 12:08:58.692728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692693    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7192c81a-2f04-48e2-b506-a8aa442efb3a" containerName="main"
Sep 11 12:08:58.692728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692716    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82fba1a6-abf3-4002-818b-eed4edceb9fd" containerName="revoke-keys"
Sep 11 12:08:58.692728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692721    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="82fba1a6-abf3-4002-818b-eed4edceb9fd" containerName="revoke-keys"
Sep 11 12:08:58.692882 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692771    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2b9a1a6-7d81-41cf-89b2-1355778569ae" containerName="payload-processing"
Sep 11 12:08:58.692882 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692780    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="82fba1a6-abf3-4002-818b-eed4edceb9fd" containerName="revoke-keys"
Sep 11 12:08:58.692882 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.692786    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7192c81a-2f04-48e2-b506-a8aa442efb3a" containerName="main"
Sep 11 12:08:58.697073 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.697058    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"
Sep 11 12:08:58.699836 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.699814    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:08:58.699985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.699818    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:08:58.703561 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.703538    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"]
Sep 11 12:08:58.809317 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.809290    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdc937c-97a9-4157-a130-8b6886072101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn\" (UID: \"fcdc937c-97a9-4157-a130-8b6886072101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"
Sep 11 12:08:58.910364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.910331    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdc937c-97a9-4157-a130-8b6886072101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn\" (UID: \"fcdc937c-97a9-4157-a130-8b6886072101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"
Sep 11 12:08:58.911031 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.911009    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdc937c-97a9-4157-a130-8b6886072101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn\" (UID: \"fcdc937c-97a9-4157-a130-8b6886072101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"
Sep 11 12:08:58.952929 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:58.952860    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:59.008662 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:59.008634    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"
Sep 11 12:08:59.280904 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:59.280835    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"]
Sep 11 12:08:59.284673 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:08:59.284645    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdc937c_97a9_4157_a130_8b6886072101.slice/crio-b743e2a1dda784795b389bc7abecdfc0e9088d8e678fb69314ca76c66ac59555 WatchSource:0}: Error finding container b743e2a1dda784795b389bc7abecdfc0e9088d8e678fb69314ca76c66ac59555: Status 404 returned error can't find the container with id b743e2a1dda784795b389bc7abecdfc0e9088d8e678fb69314ca76c66ac59555
Sep 11 12:08:59.822089 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:59.822054    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn" event={"ID":"fcdc937c-97a9-4157-a130-8b6886072101","Type":"ContainerStarted","Data":"76e3d4584c379be1debb8834b34685883dd747a639e329628e93fbc487f0a98f"}
Sep 11 12:08:59.822197 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:59.822112    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn" event={"ID":"fcdc937c-97a9-4157-a130-8b6886072101","Type":"ContainerStarted","Data":"b743e2a1dda784795b389bc7abecdfc0e9088d8e678fb69314ca76c66ac59555"}
Sep 11 12:08:59.840663 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:08:59.840619    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn" podStartSLOduration=1.8406055000000001 podStartE2EDuration="1.8406055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:59.838125277 +0000 UTC m=+1033.548796617" watchObservedRunningTime="2026-09-11 12:08:59.8406055 +0000 UTC m=+1033.551276838"
Sep 11 12:09:00.829771 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:00.829734    2586 generic.go:356] "Generic (PLEG): container finished" podID="fcdc937c-97a9-4157-a130-8b6886072101" containerID="76e3d4584c379be1debb8834b34685883dd747a639e329628e93fbc487f0a98f" exitCode=0
Sep 11 12:09:00.830144 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:00.829821    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn" event={"ID":"fcdc937c-97a9-4157-a130-8b6886072101","Type":"ContainerDied","Data":"76e3d4584c379be1debb8834b34685883dd747a639e329628e93fbc487f0a98f"}
Sep 11 12:09:01.966444 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:01.966417    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"
Sep 11 12:09:02.135260 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:02.135195    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdc937c-97a9-4157-a130-8b6886072101-maas-api-service-ca\") pod \"fcdc937c-97a9-4157-a130-8b6886072101\" (UID: \"fcdc937c-97a9-4157-a130-8b6886072101\") "
Sep 11 12:09:02.135535 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:02.135513    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcdc937c-97a9-4157-a130-8b6886072101-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcdc937c-97a9-4157-a130-8b6886072101" (UID: "fcdc937c-97a9-4157-a130-8b6886072101"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:09:02.236068 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:02.236042    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcdc937c-97a9-4157-a130-8b6886072101-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:09:02.840052 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:02.840033    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"
Sep 11 12:09:02.840225 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:02.840030    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn" event={"ID":"fcdc937c-97a9-4157-a130-8b6886072101","Type":"ContainerDied","Data":"b743e2a1dda784795b389bc7abecdfc0e9088d8e678fb69314ca76c66ac59555"}
Sep 11 12:09:02.840225 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:02.840159    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b743e2a1dda784795b389bc7abecdfc0e9088d8e678fb69314ca76c66ac59555"
Sep 11 12:09:03.088225 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:03.088154    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:03.163128 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:03.163104    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:03.247865 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:03.247840    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:03.405863 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:03.405804    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:09:03.432034 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:03.432005    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:09.130160 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.130124    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:09.442263 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.442170    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"]
Sep 11 12:09:09.442691 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.442656    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9" podUID="fc98b0fc-b130-41e5-b3a0-03d41cb52373" containerName="payload-processing" containerID="cri-o://b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891" gracePeriod=30
Sep 11 12:09:09.451063 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.450724    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"]
Sep 11 12:09:09.451162 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.451111    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw" podUID="40a093ff-8a4a-46a2-bb0d-efee9ef46768" containerName="payload-pre-processing" containerID="cri-o://604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae" gracePeriod=30
Sep 11 12:09:09.623049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.623022    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"]
Sep 11 12:09:09.625514 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.625420    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e42660-xz6kn"]
Sep 11 12:09:09.722947 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.722925    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:09:09.725617 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.725599    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:09:09.876847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.876813    2586 generic.go:356] "Generic (PLEG): container finished" podID="fc98b0fc-b130-41e5-b3a0-03d41cb52373" containerID="b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891" exitCode=0
Sep 11 12:09:09.877009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.876860    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"
Sep 11 12:09:09.877009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.876917    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9" event={"ID":"fc98b0fc-b130-41e5-b3a0-03d41cb52373","Type":"ContainerDied","Data":"b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891"}
Sep 11 12:09:09.877009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.876961    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9" event={"ID":"fc98b0fc-b130-41e5-b3a0-03d41cb52373","Type":"ContainerDied","Data":"682b9265f52fae796b8e50d9425a5b4aeb861e09cec674a6ad1a11b4170c7659"}
Sep 11 12:09:09.877009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.876987    2586 scope.go:117] "RemoveContainer" containerID="b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891"
Sep 11 12:09:09.878340 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.878323    2586 generic.go:356] "Generic (PLEG): container finished" podID="40a093ff-8a4a-46a2-bb0d-efee9ef46768" containerID="604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae" exitCode=0
Sep 11 12:09:09.878414 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.878360    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"
Sep 11 12:09:09.878414 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.878392    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw" event={"ID":"40a093ff-8a4a-46a2-bb0d-efee9ef46768","Type":"ContainerDied","Data":"604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae"}
Sep 11 12:09:09.878520 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.878423    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw" event={"ID":"40a093ff-8a4a-46a2-bb0d-efee9ef46768","Type":"ContainerDied","Data":"19192dac6ac45338c6142dd6618c08214f20c64be4ba618a03934ae4e762cede"}
Sep 11 12:09:09.885464 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.885449    2586 scope.go:117] "RemoveContainer" containerID="b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891"
Sep 11 12:09:09.885801 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:09:09.885776    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891\": container with ID starting with b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891 not found: ID does not exist" containerID="b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891"
Sep 11 12:09:09.885904 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.885804    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891"} err="failed to get container status \"b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891\": rpc error: code = NotFound desc = could not find container \"b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891\": container with ID starting with b61c2cae1940a45b028db22feabf46f6a45bd910fac70a152c56d177a6fd2891 not found: ID does not exist"
Sep 11 12:09:09.885904 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.885820    2586 scope.go:117] "RemoveContainer" containerID="604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae"
Sep 11 12:09:09.893545 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.893528    2586 scope.go:117] "RemoveContainer" containerID="604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae"
Sep 11 12:09:09.893832 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:09:09.893806    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae\": container with ID starting with 604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae not found: ID does not exist" containerID="604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae"
Sep 11 12:09:09.893908 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.893840    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae"} err="failed to get container status \"604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae\": rpc error: code = NotFound desc = could not find container \"604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae\": container with ID starting with 604f925138c9069754257d8d6db218413ef7eabb2427c30aa0038f28b162d2ae not found: ID does not exist"
Sep 11 12:09:09.897076 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.897061    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nghbl\" (UniqueName: \"kubernetes.io/projected/40a093ff-8a4a-46a2-bb0d-efee9ef46768-kube-api-access-nghbl\") pod \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") "
Sep 11 12:09:09.897137 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.897105    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a093ff-8a4a-46a2-bb0d-efee9ef46768-plugins-config-volume\") pod \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\" (UID: \"40a093ff-8a4a-46a2-bb0d-efee9ef46768\") "
Sep 11 12:09:09.897241 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.897222    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc98b0fc-b130-41e5-b3a0-03d41cb52373-plugins-config-volume\") pod \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") "
Sep 11 12:09:09.897377 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.897297    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqbgb\" (UniqueName: \"kubernetes.io/projected/fc98b0fc-b130-41e5-b3a0-03d41cb52373-kube-api-access-kqbgb\") pod \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\" (UID: \"fc98b0fc-b130-41e5-b3a0-03d41cb52373\") "
Sep 11 12:09:09.897444 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.897410    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40a093ff-8a4a-46a2-bb0d-efee9ef46768-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40a093ff-8a4a-46a2-bb0d-efee9ef46768" (UID: "40a093ff-8a4a-46a2-bb0d-efee9ef46768"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:09:09.897700 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.897681    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a093ff-8a4a-46a2-bb0d-efee9ef46768-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:09:09.897802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.897745    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc98b0fc-b130-41e5-b3a0-03d41cb52373-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc98b0fc-b130-41e5-b3a0-03d41cb52373" (UID: "fc98b0fc-b130-41e5-b3a0-03d41cb52373"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:09:09.899156 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.899137    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a093ff-8a4a-46a2-bb0d-efee9ef46768-kube-api-access-nghbl" (OuterVolumeSpecName: "kube-api-access-nghbl") pod "40a093ff-8a4a-46a2-bb0d-efee9ef46768" (UID: "40a093ff-8a4a-46a2-bb0d-efee9ef46768"). InnerVolumeSpecName "kube-api-access-nghbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:09:09.899380 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.899362    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc98b0fc-b130-41e5-b3a0-03d41cb52373-kube-api-access-kqbgb" (OuterVolumeSpecName: "kube-api-access-kqbgb") pod "fc98b0fc-b130-41e5-b3a0-03d41cb52373" (UID: "fc98b0fc-b130-41e5-b3a0-03d41cb52373"). InnerVolumeSpecName "kube-api-access-kqbgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:09:09.998193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.998138    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqbgb\" (UniqueName: \"kubernetes.io/projected/fc98b0fc-b130-41e5-b3a0-03d41cb52373-kube-api-access-kqbgb\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:09:09.998193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.998161    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nghbl\" (UniqueName: \"kubernetes.io/projected/40a093ff-8a4a-46a2-bb0d-efee9ef46768-kube-api-access-nghbl\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:09:09.998193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:09.998171    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc98b0fc-b130-41e5-b3a0-03d41cb52373-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:09:10.205126 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.205099    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"]
Sep 11 12:09:10.214894 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.209529    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e42660-6ff75f6b77-tsgz9"]
Sep 11 12:09:10.226159 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.226136    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"]
Sep 11 12:09:10.237540 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.237516    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e42660-bd9754754-nwqqw"]
Sep 11 12:09:10.681397 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.681340    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:10.783382 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.783308    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:10.823169 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.823142    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40a093ff-8a4a-46a2-bb0d-efee9ef46768" path="/var/lib/kubelet/pods/40a093ff-8a4a-46a2-bb0d-efee9ef46768/volumes"
Sep 11 12:09:10.823479 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.823465    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc98b0fc-b130-41e5-b3a0-03d41cb52373" path="/var/lib/kubelet/pods/fc98b0fc-b130-41e5-b3a0-03d41cb52373/volumes"
Sep 11 12:09:10.823787 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:10.823775    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcdc937c-97a9-4157-a130-8b6886072101" path="/var/lib/kubelet/pods/fcdc937c-97a9-4157-a130-8b6886072101/volumes"
Sep 11 12:09:13.306539 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:13.306504    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:09:13.667087 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:13.666991    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:13.760604 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:13.760573    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:14.831418 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:14.831388    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:14.891517 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:14.891476    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:14.992415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:14.992387    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:15.339029 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.338998    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:15.434262 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.434227    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:15.467234 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467210    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"]
Sep 11 12:09:15.467559 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467547    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc98b0fc-b130-41e5-b3a0-03d41cb52373" containerName="payload-processing"
Sep 11 12:09:15.467614 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467560    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc98b0fc-b130-41e5-b3a0-03d41cb52373" containerName="payload-processing"
Sep 11 12:09:15.467614 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467569    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a093ff-8a4a-46a2-bb0d-efee9ef46768" containerName="payload-pre-processing"
Sep 11 12:09:15.467614 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467574    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a093ff-8a4a-46a2-bb0d-efee9ef46768" containerName="payload-pre-processing"
Sep 11 12:09:15.467614 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467592    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdc937c-97a9-4157-a130-8b6886072101" containerName="revoke-keys"
Sep 11 12:09:15.467614 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467597    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdc937c-97a9-4157-a130-8b6886072101" containerName="revoke-keys"
Sep 11 12:09:15.467770 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467650    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdc937c-97a9-4157-a130-8b6886072101" containerName="revoke-keys"
Sep 11 12:09:15.467770 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467657    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc98b0fc-b130-41e5-b3a0-03d41cb52373" containerName="payload-processing"
Sep 11 12:09:15.467770 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.467664    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a093ff-8a4a-46a2-bb0d-efee9ef46768" containerName="payload-pre-processing"
Sep 11 12:09:15.475426 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.475405    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.478351 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.478328    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-c000cf\""
Sep 11 12:09:15.478501 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.478468    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-c000cf-dockercfg-f8hch\""
Sep 11 12:09:15.482662 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.482639    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"]
Sep 11 12:09:15.506625 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.506599    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"]
Sep 11 12:09:15.510305 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.510290    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:15.522156 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.522134    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"]
Sep 11 12:09:15.544647 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.544624    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbh8g\" (UniqueName: \"kubernetes.io/projected/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-kube-api-access-qbh8g\") pod \"payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.544775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.544698    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g4ct\" (UniqueName: \"kubernetes.io/projected/8f596b9d-641e-483c-9dd5-9d8da72ad92f-kube-api-access-4g4ct\") pod \"payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:15.544775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.544735    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.544775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.544771    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f596b9d-641e-483c-9dd5-9d8da72ad92f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:15.645642 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.645582    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f596b9d-641e-483c-9dd5-9d8da72ad92f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:15.645642 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.645636    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbh8g\" (UniqueName: \"kubernetes.io/projected/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-kube-api-access-qbh8g\") pod \"payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.645773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.645685    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g4ct\" (UniqueName: \"kubernetes.io/projected/8f596b9d-641e-483c-9dd5-9d8da72ad92f-kube-api-access-4g4ct\") pod \"payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:15.645773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.645703    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.646290 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.646271    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f596b9d-641e-483c-9dd5-9d8da72ad92f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:15.646334 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.646290    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.654721 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.654699    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbh8g\" (UniqueName: \"kubernetes.io/projected/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-kube-api-access-qbh8g\") pod \"payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.654794 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.654734    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g4ct\" (UniqueName: \"kubernetes.io/projected/8f596b9d-641e-483c-9dd5-9d8da72ad92f-kube-api-access-4g4ct\") pod \"payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:15.787060 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.787039    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:15.818969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:15.818947    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:16.062682 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.062658    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"]
Sep 11 12:09:16.064531 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:09:16.064501    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bdf33e_93cd_4804_a2fc_b4005c6ae3b6.slice/crio-5eece2654ea8a08ea26fc723c6149537847c47468ab44153aa5080104e281756 WatchSource:0}: Error finding container 5eece2654ea8a08ea26fc723c6149537847c47468ab44153aa5080104e281756: Status 404 returned error can't find the container with id 5eece2654ea8a08ea26fc723c6149537847c47468ab44153aa5080104e281756
Sep 11 12:09:16.100531 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.100506    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"]
Sep 11 12:09:16.102085 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:09:16.102058    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f596b9d_641e_483c_9dd5_9d8da72ad92f.slice/crio-aef1cda942299c84e98cc1f3e3994c6d761d28b1814393308fd1180e7b33a977 WatchSource:0}: Error finding container aef1cda942299c84e98cc1f3e3994c6d761d28b1814393308fd1180e7b33a977: Status 404 returned error can't find the container with id aef1cda942299c84e98cc1f3e3994c6d761d28b1814393308fd1180e7b33a977
Sep 11 12:09:16.551513 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.551464    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:16.652010 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.651980    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:16.843067 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.842989    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:16.917670 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.917640    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c" event={"ID":"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6","Type":"ContainerStarted","Data":"14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668"}
Sep 11 12:09:16.917891 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.917869    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c" event={"ID":"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6","Type":"ContainerStarted","Data":"5eece2654ea8a08ea26fc723c6149537847c47468ab44153aa5080104e281756"}
Sep 11 12:09:16.917971 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.917909    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:16.919256 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.919232    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9" event={"ID":"8f596b9d-641e-483c-9dd5-9d8da72ad92f","Type":"ContainerStarted","Data":"d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd"}
Sep 11 12:09:16.919351 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.919281    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9" event={"ID":"8f596b9d-641e-483c-9dd5-9d8da72ad92f","Type":"ContainerStarted","Data":"aef1cda942299c84e98cc1f3e3994c6d761d28b1814393308fd1180e7b33a977"}
Sep 11 12:09:16.919351 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.919308    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:16.944247 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.944199    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c" podStartSLOduration=1.944182461 podStartE2EDuration="1.944182461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:16.942636623 +0000 UTC m=+1050.653307962" watchObservedRunningTime="2026-09-11 12:09:16.944182461 +0000 UTC m=+1050.654853798"
Sep 11 12:09:16.963254 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:16.963214    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9" podStartSLOduration=1.963198164 podStartE2EDuration="1.963198164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:16.961770547 +0000 UTC m=+1050.672441908" watchObservedRunningTime="2026-09-11 12:09:16.963198164 +0000 UTC m=+1050.673869507"
Sep 11 12:09:17.353137 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:17.353105    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:17.654756 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:17.654668    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:17.862550 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:17.862516    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:20.740240 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:20.740205    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:20.831624 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:20.831595    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:20.907335 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:20.907299    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:21.006771 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:21.006701    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:21.132006 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:21.131969    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:21.234030 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:21.234008    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:21.954684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:21.954643    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:22.035880 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.035853    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:22.288789 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.288748    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:22.626550 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.626451    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:22.808992 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.808954    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:22.839515 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.839472    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"]
Sep 11 12:09:22.842694 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.842674    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:22.845976 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.845956    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-88968d-dockercfg-4x2wd\""
Sep 11 12:09:22.847025 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.847005    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-88968d\""
Sep 11 12:09:22.856338 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.856317    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"]
Sep 11 12:09:22.911010 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.910945    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczpl\" (UniqueName: \"kubernetes.io/projected/0ae29249-d71a-43a8-b24f-7de33b20e2f5-kube-api-access-rczpl\") pod \"payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:22.911114 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:22.911008    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae29249-d71a-43a8-b24f-7de33b20e2f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:23.012008 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.011983    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczpl\" (UniqueName: \"kubernetes.io/projected/0ae29249-d71a-43a8-b24f-7de33b20e2f5-kube-api-access-rczpl\") pod \"payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:23.012340 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.012033    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae29249-d71a-43a8-b24f-7de33b20e2f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:23.012722 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.012704    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae29249-d71a-43a8-b24f-7de33b20e2f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:23.021087 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.021068    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczpl\" (UniqueName: \"kubernetes.io/projected/0ae29249-d71a-43a8-b24f-7de33b20e2f5-kube-api-access-rczpl\") pod \"payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:23.153385 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.153365    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:23.427983 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.427962    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"]
Sep 11 12:09:23.430378 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:09:23.430351    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae29249_d71a_43a8_b24f_7de33b20e2f5.slice/crio-0eae315687e0a695419ab6f6a11ef5bead796de16ec508f05d6cb7e6853cfc42 WatchSource:0}: Error finding container 0eae315687e0a695419ab6f6a11ef5bead796de16ec508f05d6cb7e6853cfc42: Status 404 returned error can't find the container with id 0eae315687e0a695419ab6f6a11ef5bead796de16ec508f05d6cb7e6853cfc42
Sep 11 12:09:23.955215 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.955179    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx" event={"ID":"0ae29249-d71a-43a8-b24f-7de33b20e2f5","Type":"ContainerStarted","Data":"ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf"}
Sep 11 12:09:23.955454 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.955238    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx" event={"ID":"0ae29249-d71a-43a8-b24f-7de33b20e2f5","Type":"ContainerStarted","Data":"0eae315687e0a695419ab6f6a11ef5bead796de16ec508f05d6cb7e6853cfc42"}
Sep 11 12:09:23.955454 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.955322    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:23.975070 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:23.975011    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx" podStartSLOduration=1.9749959499999998 podStartE2EDuration="1.97499595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:23.973621961 +0000 UTC m=+1057.684293299" watchObservedRunningTime="2026-09-11 12:09:23.97499595 +0000 UTC m=+1057.685667293"
Sep 11 12:09:26.255778 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:26.255739    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:26.824835 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:26.824799    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:09:27.926890 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:27.926862    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:09:27.927262 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:27.926934    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:09:34.963740 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:34.963705    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:09:35.743119 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:35.743082    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:38.632872 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:38.632797    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:38.967300 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:38.967217    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:39.260891 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:39.260807    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:39.911739 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:39.911703    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:41.962133 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:41.962092    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:42.045280 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:42.045246    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:43.242687 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.242654    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:43.341051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.341023    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:43.534169 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.534070    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:43.718708 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.718675    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:43.818615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.818588    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:43.831106 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.831076    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"]
Sep 11 12:09:43.834330 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.834314    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:43.837362 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.837339    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-50e434-dockercfg-8hghw\""
Sep 11 12:09:43.837733 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.837715    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-50e434\""
Sep 11 12:09:43.845468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.845446    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"]
Sep 11 12:09:43.870182 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.870160    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"]
Sep 11 12:09:43.873516 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.873473    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:43.885250 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.885230    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"]
Sep 11 12:09:43.997045 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.997014    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35461298-3ebf-41bd-a4b1-ff7e2890e07c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:43.997185 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.997051    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c95706-a2fb-4184-8dad-0631980ad76d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:43.997185 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.997080    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwbfk\" (UniqueName: \"kubernetes.io/projected/35461298-3ebf-41bd-a4b1-ff7e2890e07c-kube-api-access-jwbfk\") pod \"payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:43.997185 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:43.997139    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng7p7\" (UniqueName: \"kubernetes.io/projected/a2c95706-a2fb-4184-8dad-0631980ad76d-kube-api-access-ng7p7\") pod \"payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:44.098274 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.098202    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c95706-a2fb-4184-8dad-0631980ad76d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:44.098274 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.098241    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwbfk\" (UniqueName: \"kubernetes.io/projected/35461298-3ebf-41bd-a4b1-ff7e2890e07c-kube-api-access-jwbfk\") pod \"payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:44.098274 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.098262    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng7p7\" (UniqueName: \"kubernetes.io/projected/a2c95706-a2fb-4184-8dad-0631980ad76d-kube-api-access-ng7p7\") pod \"payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:44.098520 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.098335    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35461298-3ebf-41bd-a4b1-ff7e2890e07c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:44.098989 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.098968    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c95706-a2fb-4184-8dad-0631980ad76d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:44.099027 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.098987    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35461298-3ebf-41bd-a4b1-ff7e2890e07c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:44.107583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.107563    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwbfk\" (UniqueName: \"kubernetes.io/projected/35461298-3ebf-41bd-a4b1-ff7e2890e07c-kube-api-access-jwbfk\") pod \"payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:44.107720 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.107704    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng7p7\" (UniqueName: \"kubernetes.io/projected/a2c95706-a2fb-4184-8dad-0631980ad76d-kube-api-access-ng7p7\") pod \"payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:44.145315 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.145286    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:44.182567 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.182536    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:44.416252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.416226    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"]
Sep 11 12:09:44.418357 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:09:44.418325    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c95706_a2fb_4184_8dad_0631980ad76d.slice/crio-c3264a490490ce3a0d7b1015afb7335bcde8d93e25b5c055f1870772906ec25d WatchSource:0}: Error finding container c3264a490490ce3a0d7b1015afb7335bcde8d93e25b5c055f1870772906ec25d: Status 404 returned error can't find the container with id c3264a490490ce3a0d7b1015afb7335bcde8d93e25b5c055f1870772906ec25d
Sep 11 12:09:44.462695 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.462676    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"]
Sep 11 12:09:44.465234 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:09:44.465201    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35461298_3ebf_41bd_a4b1_ff7e2890e07c.slice/crio-9a3fcbf5ecb7b55f3df02e9da56fd80fa9d64f4e8d6133761c106ca4b80220fd WatchSource:0}: Error finding container 9a3fcbf5ecb7b55f3df02e9da56fd80fa9d64f4e8d6133761c106ca4b80220fd: Status 404 returned error can't find the container with id 9a3fcbf5ecb7b55f3df02e9da56fd80fa9d64f4e8d6133761c106ca4b80220fd
Sep 11 12:09:44.732898 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:44.732820    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:45.061921 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.061892    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk" event={"ID":"35461298-3ebf-41bd-a4b1-ff7e2890e07c","Type":"ContainerStarted","Data":"a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f"}
Sep 11 12:09:45.062109 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.061954    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk" event={"ID":"35461298-3ebf-41bd-a4b1-ff7e2890e07c","Type":"ContainerStarted","Data":"9a3fcbf5ecb7b55f3df02e9da56fd80fa9d64f4e8d6133761c106ca4b80220fd"}
Sep 11 12:09:45.062109 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.062016    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:45.063819 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.063793    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l" event={"ID":"a2c95706-a2fb-4184-8dad-0631980ad76d","Type":"ContainerStarted","Data":"b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146"}
Sep 11 12:09:45.063926 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.063840    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l" event={"ID":"a2c95706-a2fb-4184-8dad-0631980ad76d","Type":"ContainerStarted","Data":"c3264a490490ce3a0d7b1015afb7335bcde8d93e25b5c055f1870772906ec25d"}
Sep 11 12:09:45.063926 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.063878    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:45.080962 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.080907    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk" podStartSLOduration=2.080890542 podStartE2EDuration="2.080890542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:45.080727046 +0000 UTC m=+1078.791398385" watchObservedRunningTime="2026-09-11 12:09:45.080890542 +0000 UTC m=+1078.791561880"
Sep 11 12:09:45.099268 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.099223    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l" podStartSLOduration=2.09920746 podStartE2EDuration="2.09920746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:45.09680845 +0000 UTC m=+1078.807479779" watchObservedRunningTime="2026-09-11 12:09:45.09920746 +0000 UTC m=+1078.809878798"
Sep 11 12:09:45.472606 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:45.472535    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:46.246593 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:46.246560    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:48.306721 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:48.306678    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:09:50.779649 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:50.779617    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:51.119527 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:51.119469    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:51.421393 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:51.421310    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:51.703232 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:51.703135    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:55.029452 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:55.029414    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:55.119922 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:55.119891    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:56.071447 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.071420    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:09:56.071874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.071651    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:09:56.313458 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.313424    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:56.406281 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.406203    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:56.504787 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.504764    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:56.810816 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.810767    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:56.915771 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.915732    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:56.922214 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.922191    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"]
Sep 11 12:09:56.927350 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.927335    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:56.930378 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.930359    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-4d3f83\""
Sep 11 12:09:56.930953 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.930936    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-4d3f83-dockercfg-m5rhr\""
Sep 11 12:09:56.940171 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:56.940151    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"]
Sep 11 12:09:57.111849 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.111767    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7bg4\" (UniqueName: \"kubernetes.io/projected/c586c28c-3d48-4e84-8e10-6148f6e7f84e-kube-api-access-z7bg4\") pod \"payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:57.111849 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.111812    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c586c28c-3d48-4e84-8e10-6148f6e7f84e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:57.213128 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.213094    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bg4\" (UniqueName: \"kubernetes.io/projected/c586c28c-3d48-4e84-8e10-6148f6e7f84e-kube-api-access-z7bg4\") pod \"payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:57.213286 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.213190    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c586c28c-3d48-4e84-8e10-6148f6e7f84e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:57.213976 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.213951    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c586c28c-3d48-4e84-8e10-6148f6e7f84e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:57.222947 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.222928    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bg4\" (UniqueName: \"kubernetes.io/projected/c586c28c-3d48-4e84-8e10-6148f6e7f84e-kube-api-access-z7bg4\") pod \"payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:57.237795 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.237771    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:57.509894 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:57.509870    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"]
Sep 11 12:09:57.511700 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:09:57.511674    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc586c28c_3d48_4e84_8e10_6148f6e7f84e.slice/crio-68b3f6c4bbdcf2877bb17a3884d66f1da2bbea8bd542a3e4f6da4c2096bcc157 WatchSource:0}: Error finding container 68b3f6c4bbdcf2877bb17a3884d66f1da2bbea8bd542a3e4f6da4c2096bcc157: Status 404 returned error can't find the container with id 68b3f6c4bbdcf2877bb17a3884d66f1da2bbea8bd542a3e4f6da4c2096bcc157
Sep 11 12:09:58.130166 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:58.130134    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r" event={"ID":"c586c28c-3d48-4e84-8e10-6148f6e7f84e","Type":"ContainerStarted","Data":"5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f"}
Sep 11 12:09:58.130618 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:58.130182    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r" event={"ID":"c586c28c-3d48-4e84-8e10-6148f6e7f84e","Type":"ContainerStarted","Data":"68b3f6c4bbdcf2877bb17a3884d66f1da2bbea8bd542a3e4f6da4c2096bcc157"}
Sep 11 12:09:58.130618 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:58.130226    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:09:58.148307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:09:58.148264    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r" podStartSLOduration=2.148249108 podStartE2EDuration="2.148249108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:58.146396712 +0000 UTC m=+1091.857068052" watchObservedRunningTime="2026-09-11 12:09:58.148249108 +0000 UTC m=+1091.858920449"
Sep 11 12:10:03.011597 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:03.011554    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:04.276213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:04.276178    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:04.311946 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:04.311904    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:10:06.933592 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:06.933559    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:07.680175 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.680132    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7776-4832ca"]
Sep 11 12:10:07.684080 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.684054    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:07.687174 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.687154    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-sw5w8\""
Sep 11 12:10:07.690219 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.690188    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-4832ca"]
Sep 11 12:10:07.703765 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.703734    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx46n\" (UniqueName: \"kubernetes.io/projected/74c60c1f-55d8-496b-a333-59fd1f874c6a-kube-api-access-vx46n\") pod \"test-curl-7776-4832ca\" (UID: \"74c60c1f-55d8-496b-a333-59fd1f874c6a\") " pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:07.804783 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.804747    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx46n\" (UniqueName: \"kubernetes.io/projected/74c60c1f-55d8-496b-a333-59fd1f874c6a-kube-api-access-vx46n\") pod \"test-curl-7776-4832ca\" (UID: \"74c60c1f-55d8-496b-a333-59fd1f874c6a\") " pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:07.814006 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.813975    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx46n\" (UniqueName: \"kubernetes.io/projected/74c60c1f-55d8-496b-a333-59fd1f874c6a-kube-api-access-vx46n\") pod \"test-curl-7776-4832ca\" (UID: \"74c60c1f-55d8-496b-a333-59fd1f874c6a\") " pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:07.995978 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:07.995901    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:08.267656 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:08.267582    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-4832ca"]
Sep 11 12:10:08.270628 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:08.270598    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74c60c1f_55d8_496b_a333_59fd1f874c6a.slice/crio-e09803d97b87b822d9a66ecfac0aa289a61b555e3bb22af27239bbaacebc1a83 WatchSource:0}: Error finding container e09803d97b87b822d9a66ecfac0aa289a61b555e3bb22af27239bbaacebc1a83: Status 404 returned error can't find the container with id e09803d97b87b822d9a66ecfac0aa289a61b555e3bb22af27239bbaacebc1a83
Sep 11 12:10:09.136993 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.136964    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:10:09.185063 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.185028    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-4832ca" event={"ID":"74c60c1f-55d8-496b-a333-59fd1f874c6a","Type":"ContainerStarted","Data":"37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54"}
Sep 11 12:10:09.185210 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.185073    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-4832ca" event={"ID":"74c60c1f-55d8-496b-a333-59fd1f874c6a","Type":"ContainerStarted","Data":"e09803d97b87b822d9a66ecfac0aa289a61b555e3bb22af27239bbaacebc1a83"}
Sep 11 12:10:09.200532 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.200467    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7776-4832ca" podStartSLOduration=2.20045106 podStartE2EDuration="2.20045106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:09.199477925 +0000 UTC m=+1102.910149264" watchObservedRunningTime="2026-09-11 12:10:09.20045106 +0000 UTC m=+1102.911122397"
Sep 11 12:10:09.796415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.796385    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7776-4832ca"]
Sep 11 12:10:09.802772 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.802738    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7776-4832ca"]
Sep 11 12:10:09.917447 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.917381    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7776-8e198d"]
Sep 11 12:10:09.917798 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.917786    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74c60c1f-55d8-496b-a333-59fd1f874c6a" containerName="test-curl-7776-4832ca"
Sep 11 12:10:09.917847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.917800    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="74c60c1f-55d8-496b-a333-59fd1f874c6a" containerName="test-curl-7776-4832ca"
Sep 11 12:10:09.917881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.917876    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="74c60c1f-55d8-496b-a333-59fd1f874c6a" containerName="test-curl-7776-4832ca"
Sep 11 12:10:09.920945 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.920930    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:09.925932 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:09.925913    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-8e198d"]
Sep 11 12:10:10.024721 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:10.024688    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7mv4\" (UniqueName: \"kubernetes.io/projected/63dfb703-2ec8-4f7f-97e4-d58f2d92113e-kube-api-access-r7mv4\") pod \"test-curl-7776-8e198d\" (UID: \"63dfb703-2ec8-4f7f-97e4-d58f2d92113e\") " pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:10.126042 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:10.126008    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7mv4\" (UniqueName: \"kubernetes.io/projected/63dfb703-2ec8-4f7f-97e4-d58f2d92113e-kube-api-access-r7mv4\") pod \"test-curl-7776-8e198d\" (UID: \"63dfb703-2ec8-4f7f-97e4-d58f2d92113e\") " pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:10.135188 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:10.135169    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7mv4\" (UniqueName: \"kubernetes.io/projected/63dfb703-2ec8-4f7f-97e4-d58f2d92113e-kube-api-access-r7mv4\") pod \"test-curl-7776-8e198d\" (UID: \"63dfb703-2ec8-4f7f-97e4-d58f2d92113e\") " pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:10.189437 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:10.189370    2586 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7776-4832ca" container="test-curl-7776-4832ca"
Sep 11 12:10:10.232104 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:10.232076    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:10.501463 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:10.501441    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-8e198d"]
Sep 11 12:10:10.503645 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:10.503603    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63dfb703_2ec8_4f7f_97e4_d58f2d92113e.slice/crio-e9d2c4a717a055bd9eb5df7bdaf249c2ff93d3242ab4d3a4b3ab354d5037a47b WatchSource:0}: Error finding container e9d2c4a717a055bd9eb5df7bdaf249c2ff93d3242ab4d3a4b3ab354d5037a47b: Status 404 returned error can't find the container with id e9d2c4a717a055bd9eb5df7bdaf249c2ff93d3242ab4d3a4b3ab354d5037a47b
Sep 11 12:10:11.195962 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.195912    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-8e198d" event={"ID":"63dfb703-2ec8-4f7f-97e4-d58f2d92113e","Type":"ContainerStarted","Data":"b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad"}
Sep 11 12:10:11.195962 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.195964    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-8e198d" event={"ID":"63dfb703-2ec8-4f7f-97e4-d58f2d92113e","Type":"ContainerStarted","Data":"e9d2c4a717a055bd9eb5df7bdaf249c2ff93d3242ab4d3a4b3ab354d5037a47b"}
Sep 11 12:10:11.196458 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.195982    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7776-4832ca" podUID="74c60c1f-55d8-496b-a333-59fd1f874c6a" containerName="test-curl-7776-4832ca" containerID="cri-o://37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54" gracePeriod=2
Sep 11 12:10:11.211561 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.211518    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7776-8e198d" podStartSLOduration=2.211504608 podStartE2EDuration="2.211504608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:11.209975852 +0000 UTC m=+1104.920647190" watchObservedRunningTime="2026-09-11 12:10:11.211504608 +0000 UTC m=+1104.922175951"
Sep 11 12:10:11.834511 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.834458    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7776-8e198d"]
Sep 11 12:10:11.840694 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.840663    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7776-8e198d"]
Sep 11 12:10:11.949047 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.948977    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7776-ad1e23"]
Sep 11 12:10:11.949379 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.949365    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" containerName="test-curl-7776-8e198d"
Sep 11 12:10:11.949379 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.949380    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" containerName="test-curl-7776-8e198d"
Sep 11 12:10:11.949508 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.949442    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" containerName="test-curl-7776-8e198d"
Sep 11 12:10:11.952592 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.952572    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:11.956873 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:11.956847    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-ad1e23"]
Sep 11 12:10:12.042507 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:12.042458    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smzd5\" (UniqueName: \"kubernetes.io/projected/c090a309-245c-41cb-a02f-20c01e420142-kube-api-access-smzd5\") pod \"test-curl-7776-ad1e23\" (UID: \"c090a309-245c-41cb-a02f-20c01e420142\") " pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:12.144021 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:12.143979    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smzd5\" (UniqueName: \"kubernetes.io/projected/c090a309-245c-41cb-a02f-20c01e420142-kube-api-access-smzd5\") pod \"test-curl-7776-ad1e23\" (UID: \"c090a309-245c-41cb-a02f-20c01e420142\") " pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:12.152817 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:12.152794    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smzd5\" (UniqueName: \"kubernetes.io/projected/c090a309-245c-41cb-a02f-20c01e420142-kube-api-access-smzd5\") pod \"test-curl-7776-ad1e23\" (UID: \"c090a309-245c-41cb-a02f-20c01e420142\") " pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:12.200371 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:12.200307    2586 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7776-8e198d" container="test-curl-7776-8e198d"
Sep 11 12:10:12.263172 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:12.263148    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:12.534627 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:12.534606    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-ad1e23"]
Sep 11 12:10:12.536689 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:12.536661    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc090a309_245c_41cb_a02f_20c01e420142.slice/crio-2b04437f882a1e43ab122a50449407fa575f83d6de22d45ed1f0c82072825f72 WatchSource:0}: Error finding container 2b04437f882a1e43ab122a50449407fa575f83d6de22d45ed1f0c82072825f72: Status 404 returned error can't find the container with id 2b04437f882a1e43ab122a50449407fa575f83d6de22d45ed1f0c82072825f72
Sep 11 12:10:13.207292 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.207258    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-ad1e23" event={"ID":"c090a309-245c-41cb-a02f-20c01e420142","Type":"ContainerStarted","Data":"e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34"}
Sep 11 12:10:13.207683 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.207308    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-ad1e23" event={"ID":"c090a309-245c-41cb-a02f-20c01e420142","Type":"ContainerStarted","Data":"2b04437f882a1e43ab122a50449407fa575f83d6de22d45ed1f0c82072825f72"}
Sep 11 12:10:13.207683 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.207306    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7776-8e198d" podUID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" containerName="test-curl-7776-8e198d" containerID="cri-o://b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad" gracePeriod=2
Sep 11 12:10:13.222271 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.222232    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7776-ad1e23" podStartSLOduration=2.222218186 podStartE2EDuration="2.222218186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:13.221707726 +0000 UTC m=+1106.932379066" watchObservedRunningTime="2026-09-11 12:10:13.222218186 +0000 UTC m=+1106.932889524"
Sep 11 12:10:13.388274 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.388249    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:13.456688 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.456621    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx46n\" (UniqueName: \"kubernetes.io/projected/74c60c1f-55d8-496b-a333-59fd1f874c6a-kube-api-access-vx46n\") pod \"74c60c1f-55d8-496b-a333-59fd1f874c6a\" (UID: \"74c60c1f-55d8-496b-a333-59fd1f874c6a\") "
Sep 11 12:10:13.458602 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.458577    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74c60c1f-55d8-496b-a333-59fd1f874c6a-kube-api-access-vx46n" (OuterVolumeSpecName: "kube-api-access-vx46n") pod "74c60c1f-55d8-496b-a333-59fd1f874c6a" (UID: "74c60c1f-55d8-496b-a333-59fd1f874c6a"). InnerVolumeSpecName "kube-api-access-vx46n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:10:13.558160 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.558138    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx46n\" (UniqueName: \"kubernetes.io/projected/74c60c1f-55d8-496b-a333-59fd1f874c6a-kube-api-access-vx46n\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:10:13.781969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.781882    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7776-ad1e23"]
Sep 11 12:10:13.786513 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.786452    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7776-ad1e23"]
Sep 11 12:10:13.895673 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.895647    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7776-0b8e59"]
Sep 11 12:10:13.896075 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.896061    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c090a309-245c-41cb-a02f-20c01e420142" containerName="test-curl-7776-ad1e23"
Sep 11 12:10:13.896122 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.896080    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c090a309-245c-41cb-a02f-20c01e420142" containerName="test-curl-7776-ad1e23"
Sep 11 12:10:13.896164 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.896153    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c090a309-245c-41cb-a02f-20c01e420142" containerName="test-curl-7776-ad1e23"
Sep 11 12:10:13.899141 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.899121    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:13.902620 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.902597    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-0b8e59"]
Sep 11 12:10:13.962251 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:13.962226    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl8xg\" (UniqueName: \"kubernetes.io/projected/11b44f50-2f82-455f-b21c-8fbe171595b2-kube-api-access-wl8xg\") pod \"test-curl-7776-0b8e59\" (UID: \"11b44f50-2f82-455f-b21c-8fbe171595b2\") " pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:14.063620 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.063600    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl8xg\" (UniqueName: \"kubernetes.io/projected/11b44f50-2f82-455f-b21c-8fbe171595b2-kube-api-access-wl8xg\") pod \"test-curl-7776-0b8e59\" (UID: \"11b44f50-2f82-455f-b21c-8fbe171595b2\") " pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:14.072319 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.072296    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl8xg\" (UniqueName: \"kubernetes.io/projected/11b44f50-2f82-455f-b21c-8fbe171595b2-kube-api-access-wl8xg\") pod \"test-curl-7776-0b8e59\" (UID: \"11b44f50-2f82-455f-b21c-8fbe171595b2\") " pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:14.209673 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.209650    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:14.213681 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.213654    2586 generic.go:356] "Generic (PLEG): container finished" podID="74c60c1f-55d8-496b-a333-59fd1f874c6a" containerID="37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54" exitCode=137
Sep 11 12:10:14.213778 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.213714    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:14.213778 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.213722    2586 scope.go:117] "RemoveContainer" containerID="37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54"
Sep 11 12:10:14.214073 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:14.214058    2586 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7776-ad1e23" container="test-curl-7776-ad1e23"
Sep 11 12:10:14.242460 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.242435    2586 scope.go:117] "RemoveContainer" containerID="37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54"
Sep 11 12:10:14.242825 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:14.242805    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54\": container with ID starting with 37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54 not found: ID does not exist" containerID="37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54"
Sep 11 12:10:14.242949 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.242835    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54"} err="failed to get container status \"37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54\": rpc error: code = NotFound desc = could not find container \"37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54\": container with ID starting with 37679353e296078ff2957021809248ce3bb67c18602c03a7aeea102546314e54 not found: ID does not exist"
Sep 11 12:10:14.243066 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:14.243047    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7776-4832ca\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="74c60c1f-55d8-496b-a333-59fd1f874c6a" pod="opendatahub/test-curl-7776-4832ca"
Sep 11 12:10:14.482001 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.481978    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7776-0b8e59"]
Sep 11 12:10:14.484023 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:14.483988    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11b44f50_2f82_455f_b21c_8fbe171595b2.slice/crio-5250ac66115fba4daed2a1a50095066eaed5622575c7eb086f4a97243fcde2c4 WatchSource:0}: Error finding container 5250ac66115fba4daed2a1a50095066eaed5622575c7eb086f4a97243fcde2c4: Status 404 returned error can't find the container with id 5250ac66115fba4daed2a1a50095066eaed5622575c7eb086f4a97243fcde2c4
Sep 11 12:10:14.823772 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:14.823740    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74c60c1f-55d8-496b-a333-59fd1f874c6a" path="/var/lib/kubelet/pods/74c60c1f-55d8-496b-a333-59fd1f874c6a/volumes"
Sep 11 12:10:15.221252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.221220    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-0b8e59" event={"ID":"11b44f50-2f82-455f-b21c-8fbe171595b2","Type":"ContainerStarted","Data":"d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68"}
Sep 11 12:10:15.221641 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.221269    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7776-0b8e59" event={"ID":"11b44f50-2f82-455f-b21c-8fbe171595b2","Type":"ContainerStarted","Data":"5250ac66115fba4daed2a1a50095066eaed5622575c7eb086f4a97243fcde2c4"}
Sep 11 12:10:15.221641 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.221421    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7776-ad1e23" podUID="c090a309-245c-41cb-a02f-20c01e420142" containerName="test-curl-7776-ad1e23" containerID="cri-o://e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34" gracePeriod=2
Sep 11 12:10:15.237523 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.237465    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7776-0b8e59" podStartSLOduration=2.23745142 podStartE2EDuration="2.23745142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:15.235919126 +0000 UTC m=+1108.946590465" watchObservedRunningTime="2026-09-11 12:10:15.23745142 +0000 UTC m=+1108.948122758"
Sep 11 12:10:15.364104 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.364080    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:15.476710 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.476652    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7mv4\" (UniqueName: \"kubernetes.io/projected/63dfb703-2ec8-4f7f-97e4-d58f2d92113e-kube-api-access-r7mv4\") pod \"63dfb703-2ec8-4f7f-97e4-d58f2d92113e\" (UID: \"63dfb703-2ec8-4f7f-97e4-d58f2d92113e\") "
Sep 11 12:10:15.478676 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.478656    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63dfb703-2ec8-4f7f-97e4-d58f2d92113e-kube-api-access-r7mv4" (OuterVolumeSpecName: "kube-api-access-r7mv4") pod "63dfb703-2ec8-4f7f-97e4-d58f2d92113e" (UID: "63dfb703-2ec8-4f7f-97e4-d58f2d92113e"). InnerVolumeSpecName "kube-api-access-r7mv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:10:15.577364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.577332    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7mv4\" (UniqueName: \"kubernetes.io/projected/63dfb703-2ec8-4f7f-97e4-d58f2d92113e-kube-api-access-r7mv4\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:10:15.816255 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.816230    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7776-0b8e59"]
Sep 11 12:10:15.819660 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:15.819638    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7776-0b8e59"]
Sep 11 12:10:16.227579 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.227500    2586 generic.go:356] "Generic (PLEG): container finished" podID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" containerID="b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad" exitCode=137
Sep 11 12:10:16.227579 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.227533    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:16.227579 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.227541    2586 scope.go:117] "RemoveContainer" containerID="b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad"
Sep 11 12:10:16.228048 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:16.227942    2586 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7776-0b8e59" container="test-curl-7776-0b8e59"
Sep 11 12:10:16.239690 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:16.239665    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7776-8e198d\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:16.239901 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.239886    2586 scope.go:117] "RemoveContainer" containerID="b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad"
Sep 11 12:10:16.240156 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:16.240135    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad\": container with ID starting with b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad not found: ID does not exist" containerID="b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad"
Sep 11 12:10:16.240212 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.240166    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad"} err="failed to get container status \"b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad\": rpc error: code = NotFound desc = could not find container \"b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad\": container with ID starting with b7308867b08fb5a929e152173ca718f567f4261ce84a0e17995597d4929778ad not found: ID does not exist"
Sep 11 12:10:16.393883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.393848    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:16.695214 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.695183    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:16.823107 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:16.823072    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7776-8e198d\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" pod="opendatahub/test-curl-7776-8e198d"
Sep 11 12:10:16.824045 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:16.824025    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63dfb703-2ec8-4f7f-97e4-d58f2d92113e" path="/var/lib/kubelet/pods/63dfb703-2ec8-4f7f-97e4-d58f2d92113e/volumes"
Sep 11 12:10:17.233932 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:17.233905    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7776-0b8e59" podUID="11b44f50-2f82-455f-b21c-8fbe171595b2" containerName="test-curl-7776-0b8e59" containerID="cri-o://d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68" gracePeriod=2
Sep 11 12:10:17.268885 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:17.268855    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:17.376924 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:17.376899    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:17.492713 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:17.492625    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smzd5\" (UniqueName: \"kubernetes.io/projected/c090a309-245c-41cb-a02f-20c01e420142-kube-api-access-smzd5\") pod \"c090a309-245c-41cb-a02f-20c01e420142\" (UID: \"c090a309-245c-41cb-a02f-20c01e420142\") "
Sep 11 12:10:17.494781 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:17.494750    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c090a309-245c-41cb-a02f-20c01e420142-kube-api-access-smzd5" (OuterVolumeSpecName: "kube-api-access-smzd5") pod "c090a309-245c-41cb-a02f-20c01e420142" (UID: "c090a309-245c-41cb-a02f-20c01e420142"). InnerVolumeSpecName "kube-api-access-smzd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:10:17.593671 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:17.593643    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smzd5\" (UniqueName: \"kubernetes.io/projected/c090a309-245c-41cb-a02f-20c01e420142-kube-api-access-smzd5\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:10:18.246669 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:18.246631    2586 generic.go:356] "Generic (PLEG): container finished" podID="c090a309-245c-41cb-a02f-20c01e420142" containerID="e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34" exitCode=137
Sep 11 12:10:18.247086 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:18.246679    2586 scope.go:117] "RemoveContainer" containerID="e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34"
Sep 11 12:10:18.247086 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:18.246680    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:18.255586 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:18.255568    2586 scope.go:117] "RemoveContainer" containerID="e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34"
Sep 11 12:10:18.255850 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:18.255828    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34\": container with ID starting with e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34 not found: ID does not exist" containerID="e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34"
Sep 11 12:10:18.255948 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:18.255854    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34"} err="failed to get container status \"e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34\": rpc error: code = NotFound desc = could not find container \"e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34\": container with ID starting with e9bd725783c6888f424d13e39f8fea220ff0c369a1054b5dbd69e5f571686e34 not found: ID does not exist"
Sep 11 12:10:18.257505 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:18.257455    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7776-ad1e23\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="c090a309-245c-41cb-a02f-20c01e420142" pod="opendatahub/test-curl-7776-ad1e23"
Sep 11 12:10:18.384542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:18.384449    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:18.824421 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:18.824376    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c090a309-245c-41cb-a02f-20c01e420142" path="/var/lib/kubelet/pods/c090a309-245c-41cb-a02f-20c01e420142/volumes"
Sep 11 12:10:19.425138 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:19.425115    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:19.508758 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:19.508696    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl8xg\" (UniqueName: \"kubernetes.io/projected/11b44f50-2f82-455f-b21c-8fbe171595b2-kube-api-access-wl8xg\") pod \"11b44f50-2f82-455f-b21c-8fbe171595b2\" (UID: \"11b44f50-2f82-455f-b21c-8fbe171595b2\") "
Sep 11 12:10:19.510794 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:19.510769    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b44f50-2f82-455f-b21c-8fbe171595b2-kube-api-access-wl8xg" (OuterVolumeSpecName: "kube-api-access-wl8xg") pod "11b44f50-2f82-455f-b21c-8fbe171595b2" (UID: "11b44f50-2f82-455f-b21c-8fbe171595b2"). InnerVolumeSpecName "kube-api-access-wl8xg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:10:19.609553 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:19.609512    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl8xg\" (UniqueName: \"kubernetes.io/projected/11b44f50-2f82-455f-b21c-8fbe171595b2-kube-api-access-wl8xg\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:10:19.952566 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:19.952534    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:20.218082 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.218000    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:20.258635 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.258607    2586 generic.go:356] "Generic (PLEG): container finished" podID="11b44f50-2f82-455f-b21c-8fbe171595b2" containerID="d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68" exitCode=137
Sep 11 12:10:20.258753 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.258653    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:20.258753 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.258652    2586 scope.go:117] "RemoveContainer" containerID="d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68"
Sep 11 12:10:20.267984 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.267967    2586 scope.go:117] "RemoveContainer" containerID="d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68"
Sep 11 12:10:20.268285 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:20.268267    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68\": container with ID starting with d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68 not found: ID does not exist" containerID="d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68"
Sep 11 12:10:20.268359 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.268297    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68"} err="failed to get container status \"d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68\": rpc error: code = NotFound desc = could not find container \"d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68\": container with ID starting with d1682ddfa634844ec99cc09a7e311ba9f030c1bfe7174c29fb48b5e2dff30f68 not found: ID does not exist"
Sep 11 12:10:20.269742 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:10:20.269721    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7776-0b8e59\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="11b44f50-2f82-455f-b21c-8fbe171595b2" pod="opendatahub/test-curl-7776-0b8e59"
Sep 11 12:10:20.311246 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.311223    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:20.823593 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:20.823562    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b44f50-2f82-455f-b21c-8fbe171595b2" path="/var/lib/kubelet/pods/11b44f50-2f82-455f-b21c-8fbe171595b2/volumes"
Sep 11 12:10:21.420888 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:21.420857    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:21.533014 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:21.532987    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:21.763235 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:21.763146    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:21.968097 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:21.968057    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:22.068125 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.068095    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"]
Sep 11 12:10:22.068538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.068516    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b44f50-2f82-455f-b21c-8fbe171595b2" containerName="test-curl-7776-0b8e59"
Sep 11 12:10:22.068538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.068534    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b44f50-2f82-455f-b21c-8fbe171595b2" containerName="test-curl-7776-0b8e59"
Sep 11 12:10:22.068709 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.068620    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b44f50-2f82-455f-b21c-8fbe171595b2" containerName="test-curl-7776-0b8e59"
Sep 11 12:10:22.069731 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.069707    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:22.073248 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.073233    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.076302 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.076286    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fde9d154\""
Sep 11 12:10:22.076384 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.076317    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fde9d154-dockercfg-49d7q\""
Sep 11 12:10:22.082774 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.082588    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"]
Sep 11 12:10:22.091880 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.091852    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"]
Sep 11 12:10:22.095875 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.095854    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.107550 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.107527    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"]
Sep 11 12:10:22.131845 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.131823    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6z8d\" (UniqueName: \"kubernetes.io/projected/a5b18dab-678d-4931-8a96-d19165ce5358-kube-api-access-l6z8d\") pod \"payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.131939 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.131862    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b18dab-678d-4931-8a96-d19165ce5358-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.232724 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.232689    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw7vr\" (UniqueName: \"kubernetes.io/projected/cf2a4918-4ade-4be7-9239-66f80da0fd26-kube-api-access-sw7vr\") pod \"payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") " pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.232881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.232745    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2a4918-4ade-4be7-9239-66f80da0fd26-plugins-config-volume\") pod \"payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") " pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.232881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.232769    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6z8d\" (UniqueName: \"kubernetes.io/projected/a5b18dab-678d-4931-8a96-d19165ce5358-kube-api-access-l6z8d\") pod \"payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.232881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.232799    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b18dab-678d-4931-8a96-d19165ce5358-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.233422 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.233405    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b18dab-678d-4931-8a96-d19165ce5358-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.240828 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.240801    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6z8d\" (UniqueName: \"kubernetes.io/projected/a5b18dab-678d-4931-8a96-d19165ce5358-kube-api-access-l6z8d\") pod \"payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.333806 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.333746    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2a4918-4ade-4be7-9239-66f80da0fd26-plugins-config-volume\") pod \"payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") " pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.333935 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.333816    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw7vr\" (UniqueName: \"kubernetes.io/projected/cf2a4918-4ade-4be7-9239-66f80da0fd26-kube-api-access-sw7vr\") pod \"payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") " pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.334421 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.334395    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2a4918-4ade-4be7-9239-66f80da0fd26-plugins-config-volume\") pod \"payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") " pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.341743 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.341726    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw7vr\" (UniqueName: \"kubernetes.io/projected/cf2a4918-4ade-4be7-9239-66f80da0fd26-kube-api-access-sw7vr\") pod \"payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") " pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.385545 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.385516    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:22.406782 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.406756    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:22.664039 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.664011    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"]
Sep 11 12:10:22.665572 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:22.665548    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b18dab_678d_4931_8a96_d19165ce5358.slice/crio-378871e0d2a018d4ab70dbbd401ce74f930e930f329933c8a2798aad1377286b WatchSource:0}: Error finding container 378871e0d2a018d4ab70dbbd401ce74f930e930f329933c8a2798aad1377286b: Status 404 returned error can't find the container with id 378871e0d2a018d4ab70dbbd401ce74f930e930f329933c8a2798aad1377286b
Sep 11 12:10:22.690900 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:22.690879    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"]
Sep 11 12:10:22.693520 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:22.693464    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2a4918_4ade_4be7_9239_66f80da0fd26.slice/crio-d6e6f2ac6bfc8f10e3507dd393158b8aa877dedf8c141d37cf81d2ed9c877240 WatchSource:0}: Error finding container d6e6f2ac6bfc8f10e3507dd393158b8aa877dedf8c141d37cf81d2ed9c877240: Status 404 returned error can't find the container with id d6e6f2ac6bfc8f10e3507dd393158b8aa877dedf8c141d37cf81d2ed9c877240
Sep 11 12:10:23.277549 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.277505    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv" event={"ID":"a5b18dab-678d-4931-8a96-d19165ce5358","Type":"ContainerStarted","Data":"4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4"}
Sep 11 12:10:23.277904 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.277572    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv" event={"ID":"a5b18dab-678d-4931-8a96-d19165ce5358","Type":"ContainerStarted","Data":"378871e0d2a018d4ab70dbbd401ce74f930e930f329933c8a2798aad1377286b"}
Sep 11 12:10:23.277904 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.277600    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:23.279138 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.279115    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8" event={"ID":"cf2a4918-4ade-4be7-9239-66f80da0fd26","Type":"ContainerStarted","Data":"faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14"}
Sep 11 12:10:23.279265 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.279157    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8" event={"ID":"cf2a4918-4ade-4be7-9239-66f80da0fd26","Type":"ContainerStarted","Data":"d6e6f2ac6bfc8f10e3507dd393158b8aa877dedf8c141d37cf81d2ed9c877240"}
Sep 11 12:10:23.279265 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.279205    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:23.312618 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.312569    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8" podStartSLOduration=1.312550999 podStartE2EDuration="1.312550999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:23.312050548 +0000 UTC m=+1117.022721900" watchObservedRunningTime="2026-09-11 12:10:23.312550999 +0000 UTC m=+1117.023222341"
Sep 11 12:10:23.312787 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:23.312763    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv" podStartSLOduration=1.312753842 podStartE2EDuration="1.312753842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:23.294831342 +0000 UTC m=+1117.005502722" watchObservedRunningTime="2026-09-11 12:10:23.312753842 +0000 UTC m=+1117.023425184"
Sep 11 12:10:26.399663 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:26.399628    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:10:33.494187 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.494143    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"]
Sep 11 12:10:33.499375 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.499345    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.504045 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.504022    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"openshift-service-ca.crt\""
Sep 11 12:10:33.504193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.504022    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"kube-root-ca.crt\""
Sep 11 12:10:33.504193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.504065    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"default-dockercfg-sqc5s\""
Sep 11 12:10:33.504193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.504023    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"auth-test-model-88968d-kserve-self-signed-certs\""
Sep 11 12:10:33.508762 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.508729    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"]
Sep 11 12:10:33.533623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.533593    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:33.622583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.622557    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tmp-dir\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.622681 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.622618    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-model-cache\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.622728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.622691    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr97l\" (UniqueName: \"kubernetes.io/projected/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-kube-api-access-hr97l\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.622767 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.622728    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tls-certs\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.622767 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.622752    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-home\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.622831 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.622806    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-dshm\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723516 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723469    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-home\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723611 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723530    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-dshm\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723611 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723561    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tmp-dir\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723611 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723602    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-model-cache\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723813 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723797    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-home\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723854 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723816    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr97l\" (UniqueName: \"kubernetes.io/projected/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-kube-api-access-hr97l\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723901 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723862    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tls-certs\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.723968 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723935    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tmp-dir\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.724021 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.723967    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-model-cache\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.725956 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.725929    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-dshm\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.726231 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.726213    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tls-certs\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.732170 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.732150    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr97l\" (UniqueName: \"kubernetes.io/projected/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-kube-api-access-hr97l\") pod \"auth-test-model-88968d-kserve-65d69fd9f5-t2nnr\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") " pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:33.811745 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:33.811723    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:34.082264 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:34.082239    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"]
Sep 11 12:10:34.083699 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:34.083672    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce2e9203_ef0c_4474_8a3b_4df0d9c0e811.slice/crio-b40f06dfbb2002ce9991728294f809bb03ee2c246017e2624ef0269ea9adc7da WatchSource:0}: Error finding container b40f06dfbb2002ce9991728294f809bb03ee2c246017e2624ef0269ea9adc7da: Status 404 returned error can't find the container with id b40f06dfbb2002ce9991728294f809bb03ee2c246017e2624ef0269ea9adc7da
Sep 11 12:10:34.286255 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:34.286228    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:10:34.286419 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:34.286294    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:10:34.340181 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:34.340084    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr" event={"ID":"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811","Type":"ContainerStarted","Data":"a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc"}
Sep 11 12:10:34.340181 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:34.340137    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr" event={"ID":"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811","Type":"ContainerStarted","Data":"b40f06dfbb2002ce9991728294f809bb03ee2c246017e2624ef0269ea9adc7da"}
Sep 11 12:10:34.356971 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:34.356913    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr" podStartSLOduration=1.356894625 podStartE2EDuration="1.356894625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:34.35587315 +0000 UTC m=+1128.066544489" watchObservedRunningTime="2026-09-11 12:10:34.356894625 +0000 UTC m=+1128.067565965"
Sep 11 12:10:35.722724 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:35.722690    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:35.780330 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:35.780297    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:43.812654 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:43.812617    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:43.813050 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:43.812705    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:43.825055 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:43.825033    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:44.404681 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:44.404649    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:10:45.313977 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:45.313938    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:46.912741 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:46.912711    2586 scope.go:117] "RemoveContainer" containerID="1f81c39d95cc3a8309a953317f9227fa614f85cd2ea1391e22229a461c6933d3"
Sep 11 12:10:48.593368 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:48.593322    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:10:56.503457 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.503408    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"]
Sep 11 12:10:56.506982 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.506962    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.511715 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.511693    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fde9d154\"/\"e2e-lifecycle-model-fde9d154-kserve-self-signed-certs\""
Sep 11 12:10:56.511856 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.511723    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fde9d154\"/\"openshift-service-ca.crt\""
Sep 11 12:10:56.511856 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.511750    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fde9d154\"/\"kube-root-ca.crt\""
Sep 11 12:10:56.511856 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.511693    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fde9d154\"/\"default-dockercfg-29sj7\""
Sep 11 12:10:56.515944 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.515681    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-dshm\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.515944 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.515841    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4wrq\" (UniqueName: \"kubernetes.io/projected/302278cf-8141-4246-93f2-6bbd6dbe7aab-kube-api-access-z4wrq\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.516099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.515972    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302278cf-8141-4246-93f2-6bbd6dbe7aab-tls-certs\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.516099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.516007    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-model-cache\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.516099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.516057    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-tmp-dir\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.516099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.516089    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-home\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.518160 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.518140    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"]
Sep 11 12:10:56.534318 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.534293    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:56.616776 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.616753    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4wrq\" (UniqueName: \"kubernetes.io/projected/302278cf-8141-4246-93f2-6bbd6dbe7aab-kube-api-access-z4wrq\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.616872 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.616819    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302278cf-8141-4246-93f2-6bbd6dbe7aab-tls-certs\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.616872 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.616850    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-model-cache\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.616979 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.616888    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-tmp-dir\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.616979 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.616922    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-home\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.616979 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.616965    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-dshm\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.617268 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.617248    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-model-cache\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.617335 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.617286    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-tmp-dir\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.617390 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.617369    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-home\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.619323 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.619300    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-dshm\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.619476 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.619456    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302278cf-8141-4246-93f2-6bbd6dbe7aab-tls-certs\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.626099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.626082    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4wrq\" (UniqueName: \"kubernetes.io/projected/302278cf-8141-4246-93f2-6bbd6dbe7aab-kube-api-access-z4wrq\") pod \"e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") " pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:56.816900 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:56.816879    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:10:57.092949 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:57.092920    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"]
Sep 11 12:10:57.095205 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:10:57.095172    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod302278cf_8141_4246_93f2_6bbd6dbe7aab.slice/crio-87399481d488e586ebc72f538e0f65e6983a3b6bcbaab73a7d1ff69bcdb2afb6 WatchSource:0}: Error finding container 87399481d488e586ebc72f538e0f65e6983a3b6bcbaab73a7d1ff69bcdb2afb6: Status 404 returned error can't find the container with id 87399481d488e586ebc72f538e0f65e6983a3b6bcbaab73a7d1ff69bcdb2afb6
Sep 11 12:10:57.465722 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:57.465643    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj" event={"ID":"302278cf-8141-4246-93f2-6bbd6dbe7aab","Type":"ContainerStarted","Data":"3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9"}
Sep 11 12:10:57.465722 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:57.465690    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj" event={"ID":"302278cf-8141-4246-93f2-6bbd6dbe7aab","Type":"ContainerStarted","Data":"87399481d488e586ebc72f538e0f65e6983a3b6bcbaab73a7d1ff69bcdb2afb6"}
Sep 11 12:10:57.483842 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:57.483797    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj" podStartSLOduration=1.483784135 podStartE2EDuration="1.483784135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:57.482251403 +0000 UTC m=+1151.192922744" watchObservedRunningTime="2026-09-11 12:10:57.483784135 +0000 UTC m=+1151.194455473"
Sep 11 12:10:57.966108 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:10:57.966078    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:00.209072 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:00.209036    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:11:00.689575 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:00.689532    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:06.817148 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:06.817096    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:11:06.833201 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:06.833166    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:11:06.833917 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:06.833892    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:11:07.528574 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:07.528546    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:11:15.792931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.792857    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"]
Sep 11 12:11:15.796371 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.796354    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.800706 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.800677    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"kube-root-ca.crt\""
Sep 11 12:11:15.800887 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.800714    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"openshift-service-ca.crt\""
Sep 11 12:11:15.800887 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.800832    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"default-dockercfg-6jstw\""
Sep 11 12:11:15.801044 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.800979    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"auth-test-model-4d3f83-kserve-self-signed-certs\""
Sep 11 12:11:15.804890 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.804628    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"]
Sep 11 12:11:15.867716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.867690    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-dshm\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.867855 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.867725    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tmp-dir\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.867855 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.867800    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-model-cache\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.867999 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.867896    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tls-certs\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.867999 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.867987    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9h8g\" (UniqueName: \"kubernetes.io/projected/d89af4fe-de6f-44fd-8b78-299197e1d6cc-kube-api-access-s9h8g\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.868102 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.868034    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-home\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.903640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.903615    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:15.968758 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.968735    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tls-certs\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.968873 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.968774    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9h8g\" (UniqueName: \"kubernetes.io/projected/d89af4fe-de6f-44fd-8b78-299197e1d6cc-kube-api-access-s9h8g\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.968873 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.968797    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-home\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.968873 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.968827    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-dshm\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.968873 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.968844    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tmp-dir\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.968873 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.968872    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-model-cache\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.969243 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.969221    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-home\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.969243 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.969236    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-model-cache\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.969352 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.969277    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tmp-dir\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.971132 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.971113    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-dshm\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.971373 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.971355    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tls-certs\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:15.976956 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:15.976936    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9h8g\" (UniqueName: \"kubernetes.io/projected/d89af4fe-de6f-44fd-8b78-299197e1d6cc-kube-api-access-s9h8g\") pod \"auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:16.108100 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:16.108045    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:16.381861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:16.381834    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"]
Sep 11 12:11:16.383869 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:11:16.383841    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd89af4fe_de6f_44fd_8b78_299197e1d6cc.slice/crio-97c9a5a3a3db09c9add235efccc6336f402a9e9b2e94eecb93515ee3704d9bc7 WatchSource:0}: Error finding container 97c9a5a3a3db09c9add235efccc6336f402a9e9b2e94eecb93515ee3704d9bc7: Status 404 returned error can't find the container with id 97c9a5a3a3db09c9add235efccc6336f402a9e9b2e94eecb93515ee3704d9bc7
Sep 11 12:11:16.564538 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:16.564501    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg" event={"ID":"d89af4fe-de6f-44fd-8b78-299197e1d6cc","Type":"ContainerStarted","Data":"fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e"}
Sep 11 12:11:16.564679 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:16.564549    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg" event={"ID":"d89af4fe-de6f-44fd-8b78-299197e1d6cc","Type":"ContainerStarted","Data":"97c9a5a3a3db09c9add235efccc6336f402a9e9b2e94eecb93515ee3704d9bc7"}
Sep 11 12:11:16.584454 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:16.584408    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg" podStartSLOduration=1.58439245 podStartE2EDuration="1.58439245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:11:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:11:16.582504291 +0000 UTC m=+1170.293175622" watchObservedRunningTime="2026-09-11 12:11:16.58439245 +0000 UTC m=+1170.295063788"
Sep 11 12:11:18.069502 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:18.069451    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:18.149748 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:18.149715    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:21.204788 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:21.204753    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:11:26.108785 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:26.108749    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:26.109170 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:26.108800    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:26.121132 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:26.121106    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:26.627698 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:26.627670    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:11:41.297712 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:41.297675    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:11:44.906197 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:44.906164    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:11:46.839805 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:46.839777    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:11:46.842146 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:46.842128    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:11:59.512295 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:11:59.512256    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:12:00.462671 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.462637    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:00.487574 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.487543    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"]
Sep 11 12:12:00.487823 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.487799    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8" podUID="cf2a4918-4ade-4be7-9239-66f80da0fd26" containerName="payload-processing" containerID="cri-o://faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14" gracePeriod=30
Sep 11 12:12:00.492955 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.492930    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"]
Sep 11 12:12:00.493209 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.493167    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv" podUID="a5b18dab-678d-4931-8a96-d19165ce5358" containerName="payload-pre-processing" containerID="cri-o://4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4" gracePeriod=30
Sep 11 12:12:00.527864 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.527836    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:00.647803 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.647713    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:00.760475 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.760444    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:12:00.772240 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.772207    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:00.782833 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.782788    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:12:00.795725 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.795699    2586 generic.go:356] "Generic (PLEG): container finished" podID="cf2a4918-4ade-4be7-9239-66f80da0fd26" containerID="faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14" exitCode=0
Sep 11 12:12:00.795840 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.795740    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"
Sep 11 12:12:00.795840 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.795787    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8" event={"ID":"cf2a4918-4ade-4be7-9239-66f80da0fd26","Type":"ContainerDied","Data":"faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14"}
Sep 11 12:12:00.795946 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.795868    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8" event={"ID":"cf2a4918-4ade-4be7-9239-66f80da0fd26","Type":"ContainerDied","Data":"d6e6f2ac6bfc8f10e3507dd393158b8aa877dedf8c141d37cf81d2ed9c877240"}
Sep 11 12:12:00.795946 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.795897    2586 scope.go:117] "RemoveContainer" containerID="faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14"
Sep 11 12:12:00.797182 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.797159    2586 generic.go:356] "Generic (PLEG): container finished" podID="a5b18dab-678d-4931-8a96-d19165ce5358" containerID="4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4" exitCode=0
Sep 11 12:12:00.797263 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.797196    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"
Sep 11 12:12:00.797263 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.797248    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv" event={"ID":"a5b18dab-678d-4931-8a96-d19165ce5358","Type":"ContainerDied","Data":"4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4"}
Sep 11 12:12:00.797353 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.797289    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv" event={"ID":"a5b18dab-678d-4931-8a96-d19165ce5358","Type":"ContainerDied","Data":"378871e0d2a018d4ab70dbbd401ce74f930e930f329933c8a2798aad1377286b"}
Sep 11 12:12:00.807061 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.807041    2586 scope.go:117] "RemoveContainer" containerID="faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14"
Sep 11 12:12:00.807399 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:12:00.807377    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14\": container with ID starting with faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14 not found: ID does not exist" containerID="faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14"
Sep 11 12:12:00.807524 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.807406    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14"} err="failed to get container status \"faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14\": rpc error: code = NotFound desc = could not find container \"faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14\": container with ID starting with faef13b8aad9ab8f65ad71aaadbfe30eb48226de7cc9414a934c70bbede90d14 not found: ID does not exist"
Sep 11 12:12:00.807524 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.807423    2586 scope.go:117] "RemoveContainer" containerID="4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4"
Sep 11 12:12:00.819583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.819563    2586 scope.go:117] "RemoveContainer" containerID="4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4"
Sep 11 12:12:00.819898 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:12:00.819874    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4\": container with ID starting with 4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4 not found: ID does not exist" containerID="4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4"
Sep 11 12:12:00.819984 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.819914    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4"} err="failed to get container status \"4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4\": rpc error: code = NotFound desc = could not find container \"4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4\": container with ID starting with 4b28ce7b0e543236363257db89c4f1de4376e22ce6d5afc7bab3fa909226d7f4 not found: ID does not exist"
Sep 11 12:12:00.831218 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.831192    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:00.841106 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841087    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2a4918-4ade-4be7-9239-66f80da0fd26-plugins-config-volume\") pod \"cf2a4918-4ade-4be7-9239-66f80da0fd26\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") "
Sep 11 12:12:00.841198 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841120    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6z8d\" (UniqueName: \"kubernetes.io/projected/a5b18dab-678d-4931-8a96-d19165ce5358-kube-api-access-l6z8d\") pod \"a5b18dab-678d-4931-8a96-d19165ce5358\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") "
Sep 11 12:12:00.841198 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841150    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b18dab-678d-4931-8a96-d19165ce5358-plugins-config-volume\") pod \"a5b18dab-678d-4931-8a96-d19165ce5358\" (UID: \"a5b18dab-678d-4931-8a96-d19165ce5358\") "
Sep 11 12:12:00.841270 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841199    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw7vr\" (UniqueName: \"kubernetes.io/projected/cf2a4918-4ade-4be7-9239-66f80da0fd26-kube-api-access-sw7vr\") pod \"cf2a4918-4ade-4be7-9239-66f80da0fd26\" (UID: \"cf2a4918-4ade-4be7-9239-66f80da0fd26\") "
Sep 11 12:12:00.841847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841537    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf2a4918-4ade-4be7-9239-66f80da0fd26-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf2a4918-4ade-4be7-9239-66f80da0fd26" (UID: "cf2a4918-4ade-4be7-9239-66f80da0fd26"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:12:00.841847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841596    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5b18dab-678d-4931-8a96-d19165ce5358-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5b18dab-678d-4931-8a96-d19165ce5358" (UID: "a5b18dab-678d-4931-8a96-d19165ce5358"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:12:00.841847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841734    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b18dab-678d-4931-8a96-d19165ce5358-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:00.841847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.841755    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2a4918-4ade-4be7-9239-66f80da0fd26-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:00.843300 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.843281    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b18dab-678d-4931-8a96-d19165ce5358-kube-api-access-l6z8d" (OuterVolumeSpecName: "kube-api-access-l6z8d") pod "a5b18dab-678d-4931-8a96-d19165ce5358" (UID: "a5b18dab-678d-4931-8a96-d19165ce5358"). InnerVolumeSpecName "kube-api-access-l6z8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:12:00.843435 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.843415    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf2a4918-4ade-4be7-9239-66f80da0fd26-kube-api-access-sw7vr" (OuterVolumeSpecName: "kube-api-access-sw7vr") pod "cf2a4918-4ade-4be7-9239-66f80da0fd26" (UID: "cf2a4918-4ade-4be7-9239-66f80da0fd26"). InnerVolumeSpecName "kube-api-access-sw7vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:12:00.942295 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.942273    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6z8d\" (UniqueName: \"kubernetes.io/projected/a5b18dab-678d-4931-8a96-d19165ce5358-kube-api-access-l6z8d\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:00.942295 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:00.942293    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw7vr\" (UniqueName: \"kubernetes.io/projected/cf2a4918-4ade-4be7-9239-66f80da0fd26-kube-api-access-sw7vr\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:01.120627 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:01.120605    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"]
Sep 11 12:12:01.126705 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:01.126683    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fde9d154-94857d59d-cqdcv"]
Sep 11 12:12:01.138182 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:01.138162    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"]
Sep 11 12:12:01.142442 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:01.142421    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fde9d154-64cb669df7-p29x8"]
Sep 11 12:12:02.824106 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:02.824074    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5b18dab-678d-4931-8a96-d19165ce5358" path="/var/lib/kubelet/pods/a5b18dab-678d-4931-8a96-d19165ce5358/volumes"
Sep 11 12:12:02.824499 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:02.824468    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf2a4918-4ade-4be7-9239-66f80da0fd26" path="/var/lib/kubelet/pods/cf2a4918-4ade-4be7-9239-66f80da0fd26/volumes"
Sep 11 12:12:03.833977 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:03.833946    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:08.824341 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:08.824305    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"]
Sep 11 12:12:08.875129 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:08.875086    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:17.601649 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:17.601612    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:12:23.846600 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:23.846561    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj" podUID="302278cf-8141-4246-93f2-6bbd6dbe7aab" containerName="main" containerID="cri-o://3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9" gracePeriod=45
Sep 11 12:12:24.101353 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.101276    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:12:24.248193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248158    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-dshm\") pod \"302278cf-8141-4246-93f2-6bbd6dbe7aab\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") "
Sep 11 12:12:24.248193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248197    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302278cf-8141-4246-93f2-6bbd6dbe7aab-tls-certs\") pod \"302278cf-8141-4246-93f2-6bbd6dbe7aab\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") "
Sep 11 12:12:24.248436 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248268    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-home\") pod \"302278cf-8141-4246-93f2-6bbd6dbe7aab\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") "
Sep 11 12:12:24.248436 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248311    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-model-cache\") pod \"302278cf-8141-4246-93f2-6bbd6dbe7aab\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") "
Sep 11 12:12:24.248436 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248371    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4wrq\" (UniqueName: \"kubernetes.io/projected/302278cf-8141-4246-93f2-6bbd6dbe7aab-kube-api-access-z4wrq\") pod \"302278cf-8141-4246-93f2-6bbd6dbe7aab\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") "
Sep 11 12:12:24.248624 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248443    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-tmp-dir\") pod \"302278cf-8141-4246-93f2-6bbd6dbe7aab\" (UID: \"302278cf-8141-4246-93f2-6bbd6dbe7aab\") "
Sep 11 12:12:24.248624 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248607    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-home" (OuterVolumeSpecName: "home") pod "302278cf-8141-4246-93f2-6bbd6dbe7aab" (UID: "302278cf-8141-4246-93f2-6bbd6dbe7aab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:12:24.248712 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248624    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-model-cache" (OuterVolumeSpecName: "model-cache") pod "302278cf-8141-4246-93f2-6bbd6dbe7aab" (UID: "302278cf-8141-4246-93f2-6bbd6dbe7aab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:12:24.248747 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248725    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "302278cf-8141-4246-93f2-6bbd6dbe7aab" (UID: "302278cf-8141-4246-93f2-6bbd6dbe7aab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:12:24.248885 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248865    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:24.248966 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248884    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:24.248966 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.248895    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:24.250726 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.250701    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/302278cf-8141-4246-93f2-6bbd6dbe7aab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "302278cf-8141-4246-93f2-6bbd6dbe7aab" (UID: "302278cf-8141-4246-93f2-6bbd6dbe7aab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:12:24.250842 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.250757    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-dshm" (OuterVolumeSpecName: "dshm") pod "302278cf-8141-4246-93f2-6bbd6dbe7aab" (UID: "302278cf-8141-4246-93f2-6bbd6dbe7aab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:12:24.250842 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.250760    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302278cf-8141-4246-93f2-6bbd6dbe7aab-kube-api-access-z4wrq" (OuterVolumeSpecName: "kube-api-access-z4wrq") pod "302278cf-8141-4246-93f2-6bbd6dbe7aab" (UID: "302278cf-8141-4246-93f2-6bbd6dbe7aab"). InnerVolumeSpecName "kube-api-access-z4wrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:12:24.350118 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.350073    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302278cf-8141-4246-93f2-6bbd6dbe7aab-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:24.350118 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.350115    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302278cf-8141-4246-93f2-6bbd6dbe7aab-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:24.350289 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.350129    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4wrq\" (UniqueName: \"kubernetes.io/projected/302278cf-8141-4246-93f2-6bbd6dbe7aab-kube-api-access-z4wrq\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:12:24.926942 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.926904    2586 generic.go:356] "Generic (PLEG): container finished" podID="302278cf-8141-4246-93f2-6bbd6dbe7aab" containerID="3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9" exitCode=0
Sep 11 12:12:24.926942 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.926942    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"
Sep 11 12:12:24.927461 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.926982    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj" event={"ID":"302278cf-8141-4246-93f2-6bbd6dbe7aab","Type":"ContainerDied","Data":"3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9"}
Sep 11 12:12:24.927461 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.927027    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj" event={"ID":"302278cf-8141-4246-93f2-6bbd6dbe7aab","Type":"ContainerDied","Data":"87399481d488e586ebc72f538e0f65e6983a3b6bcbaab73a7d1ff69bcdb2afb6"}
Sep 11 12:12:24.927461 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.927051    2586 scope.go:117] "RemoveContainer" containerID="3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9"
Sep 11 12:12:24.936059 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.936042    2586 scope.go:117] "RemoveContainer" containerID="3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9"
Sep 11 12:12:24.936313 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:12:24.936298    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9\": container with ID starting with 3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9 not found: ID does not exist" containerID="3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9"
Sep 11 12:12:24.936370 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.936322    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9"} err="failed to get container status \"3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9\": rpc error: code = NotFound desc = could not find container \"3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9\": container with ID starting with 3e0745baea7f3c357b4212f3d5f8c79be2cc4477f39dc7684ba6a8747cc310a9 not found: ID does not exist"
Sep 11 12:12:24.945614 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.945595    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"]
Sep 11 12:12:24.949087 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:24.949065    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fde9d154/e2e-lifecycle-model-fde9d154-kserve-cb45b85f5-97lrj"]
Sep 11 12:12:26.823831 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:26.823798    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="302278cf-8141-4246-93f2-6bbd6dbe7aab" path="/var/lib/kubelet/pods/302278cf-8141-4246-93f2-6bbd6dbe7aab/volumes"
Sep 11 12:12:32.196988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:32.196955    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:12:45.398849 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:45.398765    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:46.062606 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:46.062562    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:49.296969 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:49.296933    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:12:50.141766 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:50.141729    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:53.256402 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:53.256367    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:53.836575 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:53.836546    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:53.939077 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:53.939041    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:54.034468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:54.034429    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:12:55.290371 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:12:55.290336    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:01.029798 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:01.029768    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:01.129946 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:01.129921    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:01.690577 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:01.690530    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:07.419592 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:07.419555    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:08.196078 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:08.196046    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:13:08.233434 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:08.233403    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:08.299650 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:08.299623    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:18.087002 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:18.086949    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:18.311715 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:18.311680    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:22.102259 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:22.102226    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:13:28.406400 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:28.406368    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"]
Sep 11 12:13:28.424981 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:28.424943    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:29.415270 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:29.415234    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"]
Sep 11 12:13:29.432551 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:29.432505    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:32.909061 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909026    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"]
Sep 11 12:13:32.909513 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909479    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="302278cf-8141-4246-93f2-6bbd6dbe7aab" containerName="main"
Sep 11 12:13:32.909513 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909512    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="302278cf-8141-4246-93f2-6bbd6dbe7aab" containerName="main"
Sep 11 12:13:32.909623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909531    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5b18dab-678d-4931-8a96-d19165ce5358" containerName="payload-pre-processing"
Sep 11 12:13:32.909623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909536    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5b18dab-678d-4931-8a96-d19165ce5358" containerName="payload-pre-processing"
Sep 11 12:13:32.909623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909545    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf2a4918-4ade-4be7-9239-66f80da0fd26" containerName="payload-processing"
Sep 11 12:13:32.909623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909555    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf2a4918-4ade-4be7-9239-66f80da0fd26" containerName="payload-processing"
Sep 11 12:13:32.909623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909620    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf2a4918-4ade-4be7-9239-66f80da0fd26" containerName="payload-processing"
Sep 11 12:13:32.909802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909631    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="302278cf-8141-4246-93f2-6bbd6dbe7aab" containerName="main"
Sep 11 12:13:32.909802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.909643    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5b18dab-678d-4931-8a96-d19165ce5358" containerName="payload-pre-processing"
Sep 11 12:13:32.912976 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.912956    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:32.915843 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.915823    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"sub-test-model-88968d-kserve-self-signed-certs\""
Sep 11 12:13:32.920767 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.920746    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"]
Sep 11 12:13:32.937870 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:32.937842    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:33.040815 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.040791    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-model-cache\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.040937 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.040835    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-dshm\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.040937 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.040856    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630f476f-3617-4012-b482-647337880f45-tls-certs\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.040937 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.040920    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-home\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.041086 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.040955    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-tmp-dir\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.041086 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.040983    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh7pn\" (UniqueName: \"kubernetes.io/projected/630f476f-3617-4012-b482-647337880f45-kube-api-access-fh7pn\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.141775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.141752    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-tmp-dir\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.141894 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.141790    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh7pn\" (UniqueName: \"kubernetes.io/projected/630f476f-3617-4012-b482-647337880f45-kube-api-access-fh7pn\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.141894 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.141840    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-model-cache\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.141894 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.141871    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-dshm\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.142071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.141919    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630f476f-3617-4012-b482-647337880f45-tls-certs\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.142071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.141969    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-home\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.142179 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.142086    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-tmp-dir\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.142179 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.142167    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-model-cache\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.142261 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.142243    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-home\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.143948 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.143927    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-dshm\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.144264 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.144245    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630f476f-3617-4012-b482-647337880f45-tls-certs\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.151530 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.151504    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh7pn\" (UniqueName: \"kubernetes.io/projected/630f476f-3617-4012-b482-647337880f45-kube-api-access-fh7pn\") pod \"sub-test-model-88968d-kserve-597fd48d57-9ptms\" (UID: \"630f476f-3617-4012-b482-647337880f45\") " pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.224841 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.224791    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:33.495670 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.495646    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"]
Sep 11 12:13:33.497572 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:13:33.497546    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod630f476f_3617_4012_b482_647337880f45.slice/crio-2df904ddfe1667890848b904ea3c6859cba66070d45d7940331f566b1b37f8b1 WatchSource:0}: Error finding container 2df904ddfe1667890848b904ea3c6859cba66070d45d7940331f566b1b37f8b1: Status 404 returned error can't find the container with id 2df904ddfe1667890848b904ea3c6859cba66070d45d7940331f566b1b37f8b1
Sep 11 12:13:33.795741 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:33.795708    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:13:34.286418 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:34.286341    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms" event={"ID":"630f476f-3617-4012-b482-647337880f45","Type":"ContainerStarted","Data":"13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4"}
Sep 11 12:13:34.286418 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:34.286378    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms" event={"ID":"630f476f-3617-4012-b482-647337880f45","Type":"ContainerStarted","Data":"2df904ddfe1667890848b904ea3c6859cba66070d45d7940331f566b1b37f8b1"}
Sep 11 12:13:34.305059 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:34.305018    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms" podStartSLOduration=2.305003149 podStartE2EDuration="2.305003149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:13:34.303158 +0000 UTC m=+1308.013829342" watchObservedRunningTime="2026-09-11 12:13:34.305003149 +0000 UTC m=+1308.015674522"
Sep 11 12:13:39.208573 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:39.208535    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:40.769912 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:40.769872    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:40.870947 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:40.870909    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:43.225656 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.225629    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:43.226022 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.225699    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:43.237995 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.237975    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:43.343336 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.343316    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:13:43.425926 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.425899    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr" podUID="ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" containerName="main" containerID="cri-o://a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc" gracePeriod=45
Sep 11 12:13:43.672015 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.671992    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:13:43.726324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726296    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-home\") pod \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") "
Sep 11 12:13:43.726437 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726332    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-dshm\") pod \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") "
Sep 11 12:13:43.726437 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726400    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tmp-dir\") pod \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") "
Sep 11 12:13:43.726569 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726443    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-model-cache\") pod \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") "
Sep 11 12:13:43.726569 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726506    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tls-certs\") pod \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") "
Sep 11 12:13:43.726681 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726575    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr97l\" (UniqueName: \"kubernetes.io/projected/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-kube-api-access-hr97l\") pod \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\" (UID: \"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811\") "
Sep 11 12:13:43.726681 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726612    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-home" (OuterVolumeSpecName: "home") pod "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" (UID: "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:43.726788 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726752    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" (UID: "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:43.726788 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726768    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" (UID: "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:43.726979 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726962    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:43.727034 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726985    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:43.727034 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.726998    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:43.728740 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.728717    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-dshm" (OuterVolumeSpecName: "dshm") pod "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" (UID: "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:43.728835 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.728717    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" (UID: "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:13:43.728835 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.728790    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-kube-api-access-hr97l" (OuterVolumeSpecName: "kube-api-access-hr97l") pod "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" (UID: "ce2e9203-ef0c-4474-8a3b-4df0d9c0e811"). InnerVolumeSpecName "kube-api-access-hr97l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:13:43.827605 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.827549    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr97l\" (UniqueName: \"kubernetes.io/projected/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-kube-api-access-hr97l\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:43.827605 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.827571    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:43.827605 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:43.827581    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:44.339146 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.339119    2586 generic.go:356] "Generic (PLEG): container finished" podID="ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" containerID="a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc" exitCode=0
Sep 11 12:13:44.339570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.339164    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"
Sep 11 12:13:44.339570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.339209    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr" event={"ID":"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811","Type":"ContainerDied","Data":"a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc"}
Sep 11 12:13:44.339570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.339262    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr" event={"ID":"ce2e9203-ef0c-4474-8a3b-4df0d9c0e811","Type":"ContainerDied","Data":"b40f06dfbb2002ce9991728294f809bb03ee2c246017e2624ef0269ea9adc7da"}
Sep 11 12:13:44.339570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.339290    2586 scope.go:117] "RemoveContainer" containerID="a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc"
Sep 11 12:13:44.349021 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.349004    2586 scope.go:117] "RemoveContainer" containerID="a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc"
Sep 11 12:13:44.349287 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:13:44.349269    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc\": container with ID starting with a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc not found: ID does not exist" containerID="a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc"
Sep 11 12:13:44.349327 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.349296    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc"} err="failed to get container status \"a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc\": rpc error: code = NotFound desc = could not find container \"a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc\": container with ID starting with a796435fabfec9b8559f621f85ff6d1327f5049f00bc9bd3aa99d0d9b03d31dc not found: ID does not exist"
Sep 11 12:13:44.361575 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.361554    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"]
Sep 11 12:13:44.365607 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.365587    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/auth-test-model-88968d-kserve-65d69fd9f5-t2nnr"]
Sep 11 12:13:44.435955 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.435923    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg" podUID="d89af4fe-de6f-44fd-8b78-299197e1d6cc" containerName="main" containerID="cri-o://fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e" gracePeriod=45
Sep 11 12:13:44.694933 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.694912    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:13:44.735047 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735020    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9h8g\" (UniqueName: \"kubernetes.io/projected/d89af4fe-de6f-44fd-8b78-299197e1d6cc-kube-api-access-s9h8g\") pod \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") "
Sep 11 12:13:44.735171 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735059    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-home\") pod \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") "
Sep 11 12:13:44.735171 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735150    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-model-cache\") pod \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") "
Sep 11 12:13:44.735299 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735174    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tls-certs\") pod \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") "
Sep 11 12:13:44.735299 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735205    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tmp-dir\") pod \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") "
Sep 11 12:13:44.735299 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735232    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-dshm\") pod \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\" (UID: \"d89af4fe-de6f-44fd-8b78-299197e1d6cc\") "
Sep 11 12:13:44.735453 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735377    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-home" (OuterVolumeSpecName: "home") pod "d89af4fe-de6f-44fd-8b78-299197e1d6cc" (UID: "d89af4fe-de6f-44fd-8b78-299197e1d6cc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:44.735453 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735391    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-model-cache" (OuterVolumeSpecName: "model-cache") pod "d89af4fe-de6f-44fd-8b78-299197e1d6cc" (UID: "d89af4fe-de6f-44fd-8b78-299197e1d6cc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:44.735574 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735525    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d89af4fe-de6f-44fd-8b78-299197e1d6cc" (UID: "d89af4fe-de6f-44fd-8b78-299197e1d6cc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:44.735668 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735654    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:44.735728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735671    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:44.735728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.735679    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:44.737454 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.737428    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-dshm" (OuterVolumeSpecName: "dshm") pod "d89af4fe-de6f-44fd-8b78-299197e1d6cc" (UID: "d89af4fe-de6f-44fd-8b78-299197e1d6cc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:13:44.737616 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.737514    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d89af4fe-de6f-44fd-8b78-299197e1d6cc" (UID: "d89af4fe-de6f-44fd-8b78-299197e1d6cc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:13:44.737715 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.737697    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89af4fe-de6f-44fd-8b78-299197e1d6cc-kube-api-access-s9h8g" (OuterVolumeSpecName: "kube-api-access-s9h8g") pod "d89af4fe-de6f-44fd-8b78-299197e1d6cc" (UID: "d89af4fe-de6f-44fd-8b78-299197e1d6cc"). InnerVolumeSpecName "kube-api-access-s9h8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:13:44.824260 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.824234    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" path="/var/lib/kubelet/pods/ce2e9203-ef0c-4474-8a3b-4df0d9c0e811/volumes"
Sep 11 12:13:44.836242 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.836223    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9h8g\" (UniqueName: \"kubernetes.io/projected/d89af4fe-de6f-44fd-8b78-299197e1d6cc-kube-api-access-s9h8g\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:44.836333 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.836243    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d89af4fe-de6f-44fd-8b78-299197e1d6cc-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:44.836333 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:44.836252    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d89af4fe-de6f-44fd-8b78-299197e1d6cc-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:13:45.135287 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.135258    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:45.227926 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.227899    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:45.347012 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.346986    2586 generic.go:356] "Generic (PLEG): container finished" podID="d89af4fe-de6f-44fd-8b78-299197e1d6cc" containerID="fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e" exitCode=0
Sep 11 12:13:45.347394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.347041    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"
Sep 11 12:13:45.347394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.347066    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg" event={"ID":"d89af4fe-de6f-44fd-8b78-299197e1d6cc","Type":"ContainerDied","Data":"fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e"}
Sep 11 12:13:45.347394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.347112    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg" event={"ID":"d89af4fe-de6f-44fd-8b78-299197e1d6cc","Type":"ContainerDied","Data":"97c9a5a3a3db09c9add235efccc6336f402a9e9b2e94eecb93515ee3704d9bc7"}
Sep 11 12:13:45.347394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.347134    2586 scope.go:117] "RemoveContainer" containerID="fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e"
Sep 11 12:13:45.355856 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.355841    2586 scope.go:117] "RemoveContainer" containerID="fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e"
Sep 11 12:13:45.356135 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:13:45.356114    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e\": container with ID starting with fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e not found: ID does not exist" containerID="fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e"
Sep 11 12:13:45.356237 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.356141    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e"} err="failed to get container status \"fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e\": rpc error: code = NotFound desc = could not find container \"fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e\": container with ID starting with fd814310b9c31c857e70d5d47e29e3d5ce1d25193020f36f0807337981bb953e not found: ID does not exist"
Sep 11 12:13:45.364329 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.364307    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"]
Sep 11 12:13:45.369386 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.369368    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/auth-test-model-4d3f83-kserve-6f6f4d4b87-xlrfg"]
Sep 11 12:13:45.451091 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.451040    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:45.745859 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.745781    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:45.856682 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:45.856654    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:46.823810 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:46.823783    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d89af4fe-de6f-44fd-8b78-299197e1d6cc" path="/var/lib/kubelet/pods/d89af4fe-de6f-44fd-8b78-299197e1d6cc/volumes"
Sep 11 12:13:47.495548 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:47.495519    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:13:48.513923 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:48.513890    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:48.611431 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:48.611407    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:49.860589 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:49.860553    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:49.922101 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:49.922077    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:50.246572 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.246478    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:50.497906 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.497828    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:50.594684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.594638    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:13:50.595864 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.595837    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"]
Sep 11 12:13:50.596263 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.596247    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89af4fe-de6f-44fd-8b78-299197e1d6cc" containerName="main"
Sep 11 12:13:50.596337 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.596267    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89af4fe-de6f-44fd-8b78-299197e1d6cc" containerName="main"
Sep 11 12:13:50.596337 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.596295    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" containerName="main"
Sep 11 12:13:50.596337 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.596304    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" containerName="main"
Sep 11 12:13:50.596516 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.596413    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89af4fe-de6f-44fd-8b78-299197e1d6cc" containerName="main"
Sep 11 12:13:50.596516 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.596431    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce2e9203-ef0c-4474-8a3b-4df0d9c0e811" containerName="main"
Sep 11 12:13:50.601042 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.601021    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.604093 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.604073    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-482cb0-dockercfg-x6kns\""
Sep 11 12:13:50.604093 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.604086    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-482cb0\""
Sep 11 12:13:50.608662 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.608575    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"]
Sep 11 12:13:50.630710 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.630688    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"]
Sep 11 12:13:50.634084 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.634066    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:50.649056 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.649035    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"]
Sep 11 12:13:50.685194 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.685169    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-plugins-config-volume\") pod \"payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") " pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:50.685300 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.685207    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxlj9\" (UniqueName: \"kubernetes.io/projected/26fd313e-217a-4b7e-a3be-0a6c0129d7af-kube-api-access-mxlj9\") pod \"payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.685300 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.685236    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qxgj\" (UniqueName: \"kubernetes.io/projected/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-kube-api-access-7qxgj\") pod \"payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") " pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:50.685392 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.685308    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fd313e-217a-4b7e-a3be-0a6c0129d7af-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.786280 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.786220    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fd313e-217a-4b7e-a3be-0a6c0129d7af-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.786373 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.786295    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-plugins-config-volume\") pod \"payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") " pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:50.786373 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.786328    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxlj9\" (UniqueName: \"kubernetes.io/projected/26fd313e-217a-4b7e-a3be-0a6c0129d7af-kube-api-access-mxlj9\") pod \"payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.786523 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.786372    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qxgj\" (UniqueName: \"kubernetes.io/projected/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-kube-api-access-7qxgj\") pod \"payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") " pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:50.786984 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.786963    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fd313e-217a-4b7e-a3be-0a6c0129d7af-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.787072 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.787056    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-plugins-config-volume\") pod \"payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") " pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:50.796169 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.796141    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qxgj\" (UniqueName: \"kubernetes.io/projected/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-kube-api-access-7qxgj\") pod \"payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") " pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:50.796257 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.796225    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxlj9\" (UniqueName: \"kubernetes.io/projected/26fd313e-217a-4b7e-a3be-0a6c0129d7af-kube-api-access-mxlj9\") pod \"payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.912579 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.912554    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:50.943644 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:50.943599    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:51.188713 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.188664    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"]
Sep 11 12:13:51.190934 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:13:51.190903    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26fd313e_217a_4b7e_a3be_0a6c0129d7af.slice/crio-12f8ab7540c37a595f140b99848a4c56102a96181c78c0dffaf40cbf571e5cf2 WatchSource:0}: Error finding container 12f8ab7540c37a595f140b99848a4c56102a96181c78c0dffaf40cbf571e5cf2: Status 404 returned error can't find the container with id 12f8ab7540c37a595f140b99848a4c56102a96181c78c0dffaf40cbf571e5cf2
Sep 11 12:13:51.227372 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.227353    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"]
Sep 11 12:13:51.228622 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:13:51.228600    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0a8bf7_3ae9_4858_8891_2defc1ec8360.slice/crio-94c403d37757cab74e7abfb0c1fd5b2d7fbfdf4b981bb7e7ae1f78b868967a4a WatchSource:0}: Error finding container 94c403d37757cab74e7abfb0c1fd5b2d7fbfdf4b981bb7e7ae1f78b868967a4a: Status 404 returned error can't find the container with id 94c403d37757cab74e7abfb0c1fd5b2d7fbfdf4b981bb7e7ae1f78b868967a4a
Sep 11 12:13:51.378660 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.378605    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp" event={"ID":"26fd313e-217a-4b7e-a3be-0a6c0129d7af","Type":"ContainerStarted","Data":"613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8"}
Sep 11 12:13:51.378883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.378679    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:13:51.378883 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.378706    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp" event={"ID":"26fd313e-217a-4b7e-a3be-0a6c0129d7af","Type":"ContainerStarted","Data":"12f8ab7540c37a595f140b99848a4c56102a96181c78c0dffaf40cbf571e5cf2"}
Sep 11 12:13:51.380322 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.380286    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl" event={"ID":"0f0a8bf7-3ae9-4858-8891-2defc1ec8360","Type":"ContainerStarted","Data":"9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977"}
Sep 11 12:13:51.380403 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.380339    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl" event={"ID":"0f0a8bf7-3ae9-4858-8891-2defc1ec8360","Type":"ContainerStarted","Data":"94c403d37757cab74e7abfb0c1fd5b2d7fbfdf4b981bb7e7ae1f78b868967a4a"}
Sep 11 12:13:51.380403 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.380368    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:13:51.396746 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.396695    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp" podStartSLOduration=1.396681351 podStartE2EDuration="1.396681351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:13:51.39445756 +0000 UTC m=+1325.105128909" watchObservedRunningTime="2026-09-11 12:13:51.396681351 +0000 UTC m=+1325.107352734"
Sep 11 12:13:51.412756 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:13:51.412715    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl" podStartSLOduration=1.412700858 podStartE2EDuration="1.412700858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:13:51.411128684 +0000 UTC m=+1325.121800026" watchObservedRunningTime="2026-09-11 12:13:51.412700858 +0000 UTC m=+1325.123372196"
Sep 11 12:14:02.387714 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:02.387678    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:14:02.388286 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:02.387769    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:14:05.897508 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:05.897461    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:14:10.121053 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:10.120940    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:10.350701 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:10.350666    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:13.444374 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:13.444342    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:13.898774 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:13.898739    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:14.211643 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.211560    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:14.670424 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.670392    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:14.890379 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.890352    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"]
Sep 11 12:14:14.895181 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.895159    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:14.899644 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.899625    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"sub-test-model-4d3f83-kserve-self-signed-certs\""
Sep 11 12:14:14.899644 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.899638    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"openshift-service-ca.crt\""
Sep 11 12:14:14.899644 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.899625    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"default-dockercfg-6jstw\""
Sep 11 12:14:14.899857 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.899686    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"kube-root-ca.crt\""
Sep 11 12:14:14.904026 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.904003    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"]
Sep 11 12:14:14.919967 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.919943    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:14.985814 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.985757    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29cb4fa1-a19f-4764-a486-f8c31370edf4-tls-certs\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:14.985814 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.985784    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-home\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:14.985814 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.985801    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-tmp-dir\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:14.986000 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.985826    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-dshm\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:14.986000 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.985932    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkkdn\" (UniqueName: \"kubernetes.io/projected/29cb4fa1-a19f-4764-a486-f8c31370edf4-kube-api-access-vkkdn\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:14.986000 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:14.985970    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-model-cache\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.086414 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086387    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-dshm\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.086578 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086453    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkkdn\" (UniqueName: \"kubernetes.io/projected/29cb4fa1-a19f-4764-a486-f8c31370edf4-kube-api-access-vkkdn\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.086578 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086479    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-model-cache\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.086704 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086615    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29cb4fa1-a19f-4764-a486-f8c31370edf4-tls-certs\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.086704 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086644    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-home\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.086704 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086666    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-tmp-dir\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.086932 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086882    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-model-cache\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.087056 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086935    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-home\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.087056 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.086994    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-tmp-dir\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.088565 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.088549    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-dshm\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.089082 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.089067    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29cb4fa1-a19f-4764-a486-f8c31370edf4-tls-certs\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.097062 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.097044    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkkdn\" (UniqueName: \"kubernetes.io/projected/29cb4fa1-a19f-4764-a486-f8c31370edf4-kube-api-access-vkkdn\") pod \"sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.206868 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.206849    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:15.479783 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.479649    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"]
Sep 11 12:14:15.482214 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:14:15.482187    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29cb4fa1_a19f_4764_a486_f8c31370edf4.slice/crio-7a7e97f5160bec3afab4d1ff8d98b13016af859c1d402318cbbb7b0682ed87bf WatchSource:0}: Error finding container 7a7e97f5160bec3afab4d1ff8d98b13016af859c1d402318cbbb7b0682ed87bf: Status 404 returned error can't find the container with id 7a7e97f5160bec3afab4d1ff8d98b13016af859c1d402318cbbb7b0682ed87bf
Sep 11 12:14:15.506399 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:15.506351    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6" event={"ID":"29cb4fa1-a19f-4764-a486-f8c31370edf4","Type":"ContainerStarted","Data":"7a7e97f5160bec3afab4d1ff8d98b13016af859c1d402318cbbb7b0682ed87bf"}
Sep 11 12:14:16.513340 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:16.513304    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6" event={"ID":"29cb4fa1-a19f-4764-a486-f8c31370edf4","Type":"ContainerStarted","Data":"4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1"}
Sep 11 12:14:16.533204 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:16.533146    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6" podStartSLOduration=2.5331332250000003 podStartE2EDuration="2.533133225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:16.531198274 +0000 UTC m=+1350.241869611" watchObservedRunningTime="2026-09-11 12:14:16.533133225 +0000 UTC m=+1350.243804560"
Sep 11 12:14:17.748008 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:17.747971    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:17.849129 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:17.849067    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:18.934905 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:18.934866    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:19.034722 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:19.034687    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:19.134491 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:19.134464    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:19.515571 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:19.515541    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:19.601168 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:19.601133    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:20.013292 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:20.013252    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:14:25.207570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:25.207534    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:25.207941 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:25.207738    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:25.219800 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:25.219775    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:25.573242 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:25.573216    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:14:36.602823 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:36.602789    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:14:36.670947 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:36.670918    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:39.917697 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:39.917659    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:42.705112 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:42.705079    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:45.435081 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:45.435051    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:46.153292 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:46.153262    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:14:46.986025 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:46.985996    2586 scope.go:117] "RemoveContainer" containerID="0cc55a08a6f84dd7d9c3bde8379a0cf0dc2eb58b00e1fdc14b6592c876b96fcd"
Sep 11 12:14:55.602279 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:14:55.602243    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:15:00.151944 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.151914    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"]
Sep 11 12:15:00.156944 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.156927    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"]
Sep 11 12:15:00.157064 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.157045    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"
Sep 11 12:15:00.160206 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.160183    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:15:00.160598 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.160580    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"
Sep 11 12:15:00.169174 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.169158    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"]
Sep 11 12:15:00.171822 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.171724    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"]
Sep 11 12:15:00.172332 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.172311    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"
Sep 11 12:15:00.180576 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.180551    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"]
Sep 11 12:15:00.181115 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.181098    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"
Sep 11 12:15:00.184668 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.184651    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"
Sep 11 12:15:00.203772 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.203745    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"]
Sep 11 12:15:00.234072 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.234052    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"
Sep 11 12:15:00.464294 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.464217    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"]
Sep 11 12:15:00.468787 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:15:00.468760    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod360e8251_5c30_4d42_ac76_eed34e74b168.slice/crio-9e139f2070dc8365327b0c14c8c843069231b72002f7e5e8a3ced4176d16b0dd WatchSource:0}: Error finding container 9e139f2070dc8365327b0c14c8c843069231b72002f7e5e8a3ced4176d16b0dd: Status 404 returned error can't find the container with id 9e139f2070dc8365327b0c14c8c843069231b72002f7e5e8a3ced4176d16b0dd
Sep 11 12:15:00.485519 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.478173    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"]
Sep 11 12:15:00.485519 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:15:00.481572    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efd514b_7d02_4fdb_8170_4f0fd3f82f14.slice/crio-e15ef388cac292bf04e209b5c12a41f60c5b0c62725f0542e74320a951425da9 WatchSource:0}: Error finding container e15ef388cac292bf04e209b5c12a41f60c5b0c62725f0542e74320a951425da9: Status 404 returned error can't find the container with id e15ef388cac292bf04e209b5c12a41f60c5b0c62725f0542e74320a951425da9
Sep 11 12:15:00.525815 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.525796    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"]
Sep 11 12:15:00.527499 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:15:00.527457    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9913708_b6f6_4690_ac4a_fd2022bdfe36.slice/crio-f0d2e55c6e05507208759e61a3276f88fb6e3eaa8b8e9323d420b1e1a5cbce1d WatchSource:0}: Error finding container f0d2e55c6e05507208759e61a3276f88fb6e3eaa8b8e9323d420b1e1a5cbce1d: Status 404 returned error can't find the container with id f0d2e55c6e05507208759e61a3276f88fb6e3eaa8b8e9323d420b1e1a5cbce1d
Sep 11 12:15:00.741086 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.741060    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8" event={"ID":"8efd514b-7d02-4fdb-8170-4f0fd3f82f14","Type":"ContainerStarted","Data":"e15ef388cac292bf04e209b5c12a41f60c5b0c62725f0542e74320a951425da9"}
Sep 11 12:15:00.742370 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.742349    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn" event={"ID":"360e8251-5c30-4d42-ac76-eed34e74b168","Type":"ContainerStarted","Data":"9e139f2070dc8365327b0c14c8c843069231b72002f7e5e8a3ced4176d16b0dd"}
Sep 11 12:15:00.743493 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.743464    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq" event={"ID":"f9913708-b6f6-4690-ac4a-fd2022bdfe36","Type":"ContainerStarted","Data":"f0d2e55c6e05507208759e61a3276f88fb6e3eaa8b8e9323d420b1e1a5cbce1d"}
Sep 11 12:15:00.936703 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.936669    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"]
Sep 11 12:15:00.975972 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.975939    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"]
Sep 11 12:15:00.976214 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.976184    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl" podUID="0f0a8bf7-3ae9-4858-8891-2defc1ec8360" containerName="payload-processing" containerID="cri-o://9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977" gracePeriod=30
Sep 11 12:15:00.985081 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.985056    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"]
Sep 11 12:15:00.985294 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:00.985271    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp" podUID="26fd313e-217a-4b7e-a3be-0a6c0129d7af" containerName="payload-pre-processing" containerID="cri-o://613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8" gracePeriod=30
Sep 11 12:15:01.239635 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.239579    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:15:01.270226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.270205    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:15:01.270337 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.270203    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qxgj\" (UniqueName: \"kubernetes.io/projected/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-kube-api-access-7qxgj\") pod \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") "
Sep 11 12:15:01.270409 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.270357    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-plugins-config-volume\") pod \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\" (UID: \"0f0a8bf7-3ae9-4858-8891-2defc1ec8360\") "
Sep 11 12:15:01.270769 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.270748    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f0a8bf7-3ae9-4858-8891-2defc1ec8360" (UID: "0f0a8bf7-3ae9-4858-8891-2defc1ec8360"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:01.272281 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.272262    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-kube-api-access-7qxgj" (OuterVolumeSpecName: "kube-api-access-7qxgj") pod "0f0a8bf7-3ae9-4858-8891-2defc1ec8360" (UID: "0f0a8bf7-3ae9-4858-8891-2defc1ec8360"). InnerVolumeSpecName "kube-api-access-7qxgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:01.371324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.371293    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxlj9\" (UniqueName: \"kubernetes.io/projected/26fd313e-217a-4b7e-a3be-0a6c0129d7af-kube-api-access-mxlj9\") pod \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") "
Sep 11 12:15:01.371448 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.371362    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fd313e-217a-4b7e-a3be-0a6c0129d7af-plugins-config-volume\") pod \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\" (UID: \"26fd313e-217a-4b7e-a3be-0a6c0129d7af\") "
Sep 11 12:15:01.371686 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.371668    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qxgj\" (UniqueName: \"kubernetes.io/projected/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-kube-api-access-7qxgj\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:01.371745 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.371690    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0a8bf7-3ae9-4858-8891-2defc1ec8360-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:01.371745 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.371690    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26fd313e-217a-4b7e-a3be-0a6c0129d7af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26fd313e-217a-4b7e-a3be-0a6c0129d7af" (UID: "26fd313e-217a-4b7e-a3be-0a6c0129d7af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:01.373426 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.373407    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26fd313e-217a-4b7e-a3be-0a6c0129d7af-kube-api-access-mxlj9" (OuterVolumeSpecName: "kube-api-access-mxlj9") pod "26fd313e-217a-4b7e-a3be-0a6c0129d7af" (UID: "26fd313e-217a-4b7e-a3be-0a6c0129d7af"). InnerVolumeSpecName "kube-api-access-mxlj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:01.472876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.472852    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxlj9\" (UniqueName: \"kubernetes.io/projected/26fd313e-217a-4b7e-a3be-0a6c0129d7af-kube-api-access-mxlj9\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:01.472876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.472873    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fd313e-217a-4b7e-a3be-0a6c0129d7af-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:01.548230 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.548202    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:01.642210 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.642179    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:01.750389 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.750366    2586 generic.go:356] "Generic (PLEG): container finished" podID="f9913708-b6f6-4690-ac4a-fd2022bdfe36" containerID="f60e6e3569b4f430b33548e700dbaecf7007ce433a50e91d1dc4ecd90bcfb246" exitCode=0
Sep 11 12:15:01.750517 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.750459    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq" event={"ID":"f9913708-b6f6-4690-ac4a-fd2022bdfe36","Type":"ContainerDied","Data":"f60e6e3569b4f430b33548e700dbaecf7007ce433a50e91d1dc4ecd90bcfb246"}
Sep 11 12:15:01.751783 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.751762    2586 generic.go:356] "Generic (PLEG): container finished" podID="26fd313e-217a-4b7e-a3be-0a6c0129d7af" containerID="613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8" exitCode=0
Sep 11 12:15:01.751881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.751820    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"
Sep 11 12:15:01.751881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.751843    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp" event={"ID":"26fd313e-217a-4b7e-a3be-0a6c0129d7af","Type":"ContainerDied","Data":"613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8"}
Sep 11 12:15:01.751992 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.751888    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp" event={"ID":"26fd313e-217a-4b7e-a3be-0a6c0129d7af","Type":"ContainerDied","Data":"12f8ab7540c37a595f140b99848a4c56102a96181c78c0dffaf40cbf571e5cf2"}
Sep 11 12:15:01.751992 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.751909    2586 scope.go:117] "RemoveContainer" containerID="613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8"
Sep 11 12:15:01.753238 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.753217    2586 generic.go:356] "Generic (PLEG): container finished" podID="0f0a8bf7-3ae9-4858-8891-2defc1ec8360" containerID="9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977" exitCode=0
Sep 11 12:15:01.753339 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.753286    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"
Sep 11 12:15:01.753339 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.753300    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl" event={"ID":"0f0a8bf7-3ae9-4858-8891-2defc1ec8360","Type":"ContainerDied","Data":"9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977"}
Sep 11 12:15:01.753456 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.753360    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl" event={"ID":"0f0a8bf7-3ae9-4858-8891-2defc1ec8360","Type":"ContainerDied","Data":"94c403d37757cab74e7abfb0c1fd5b2d7fbfdf4b981bb7e7ae1f78b868967a4a"}
Sep 11 12:15:01.755260 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.755239    2586 generic.go:356] "Generic (PLEG): container finished" podID="8efd514b-7d02-4fdb-8170-4f0fd3f82f14" containerID="6f00734c7343634c38e25b4c7971841d483a4be45467a1d8d9625d38f351dc76" exitCode=0
Sep 11 12:15:01.755361 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.755301    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8" event={"ID":"8efd514b-7d02-4fdb-8170-4f0fd3f82f14","Type":"ContainerDied","Data":"6f00734c7343634c38e25b4c7971841d483a4be45467a1d8d9625d38f351dc76"}
Sep 11 12:15:01.756965 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.756944    2586 generic.go:356] "Generic (PLEG): container finished" podID="360e8251-5c30-4d42-ac76-eed34e74b168" containerID="209f5cef0a95c21dbc7fd190c398554ad2b2c9105fdfac4da26aa4a4d79daeac" exitCode=6
Sep 11 12:15:01.757047 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.756992    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn" event={"ID":"360e8251-5c30-4d42-ac76-eed34e74b168","Type":"ContainerDied","Data":"209f5cef0a95c21dbc7fd190c398554ad2b2c9105fdfac4da26aa4a4d79daeac"}
Sep 11 12:15:01.757096 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.757067    2586 scope.go:117] "RemoveContainer" containerID="209f5cef0a95c21dbc7fd190c398554ad2b2c9105fdfac4da26aa4a4d79daeac"
Sep 11 12:15:01.774145 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.774126    2586 scope.go:117] "RemoveContainer" containerID="613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8"
Sep 11 12:15:01.774455 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:01.774435    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8\": container with ID starting with 613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8 not found: ID does not exist" containerID="613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8"
Sep 11 12:15:01.774571 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.774466    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8"} err="failed to get container status \"613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8\": rpc error: code = NotFound desc = could not find container \"613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8\": container with ID starting with 613eda60bd88a6e91f503c4183840a0e609883abcfb4c3f5f84cce83734edff8 not found: ID does not exist"
Sep 11 12:15:01.774571 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.774551    2586 scope.go:117] "RemoveContainer" containerID="9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977"
Sep 11 12:15:01.790721 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.790697    2586 scope.go:117] "RemoveContainer" containerID="9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977"
Sep 11 12:15:01.791001 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:01.790982    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977\": container with ID starting with 9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977 not found: ID does not exist" containerID="9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977"
Sep 11 12:15:01.791068 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.791007    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977"} err="failed to get container status \"9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977\": rpc error: code = NotFound desc = could not find container \"9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977\": container with ID starting with 9f933cf0d3415359f8323a5fb3f30303bfce8007a26eadb2c4a4c5eb2f35b977 not found: ID does not exist"
Sep 11 12:15:01.820877 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.820824    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"]
Sep 11 12:15:01.822745 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.822713    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-482cb0-6d7467d5fd-z6btp"]
Sep 11 12:15:01.833586 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.833561    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"]
Sep 11 12:15:01.838124 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:01.838103    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-482cb0-6f747fb949-ps8cl"]
Sep 11 12:15:02.764637 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.764609    2586 generic.go:356] "Generic (PLEG): container finished" podID="360e8251-5c30-4d42-ac76-eed34e74b168" containerID="3ef1ee3438177fe67a27bf5a8023485ff55794998e2601900ed138ddc850962a" exitCode=6
Sep 11 12:15:02.765037 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.764685    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn" event={"ID":"360e8251-5c30-4d42-ac76-eed34e74b168","Type":"ContainerDied","Data":"3ef1ee3438177fe67a27bf5a8023485ff55794998e2601900ed138ddc850962a"}
Sep 11 12:15:02.765037 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.764738    2586 scope.go:117] "RemoveContainer" containerID="209f5cef0a95c21dbc7fd190c398554ad2b2c9105fdfac4da26aa4a4d79daeac"
Sep 11 12:15:02.825588 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.825555    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0a8bf7-3ae9-4858-8891-2defc1ec8360" path="/var/lib/kubelet/pods/0f0a8bf7-3ae9-4858-8891-2defc1ec8360/volumes"
Sep 11 12:15:02.826050 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.826029    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26fd313e-217a-4b7e-a3be-0a6c0129d7af" path="/var/lib/kubelet/pods/26fd313e-217a-4b7e-a3be-0a6c0129d7af/volumes"
Sep 11 12:15:02.951928 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.951908    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"
Sep 11 12:15:02.953964 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.953945    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"
Sep 11 12:15:02.957394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:02.957377    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"
Sep 11 12:15:03.772502 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.772456    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"
Sep 11 12:15:03.772502 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.772464    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8" event={"ID":"8efd514b-7d02-4fdb-8170-4f0fd3f82f14","Type":"ContainerDied","Data":"e15ef388cac292bf04e209b5c12a41f60c5b0c62725f0542e74320a951425da9"}
Sep 11 12:15:03.772985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.772533    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e15ef388cac292bf04e209b5c12a41f60c5b0c62725f0542e74320a951425da9"
Sep 11 12:15:03.773943 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.773924    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"
Sep 11 12:15:03.774104 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.773976    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn" event={"ID":"360e8251-5c30-4d42-ac76-eed34e74b168","Type":"ContainerDied","Data":"9e139f2070dc8365327b0c14c8c843069231b72002f7e5e8a3ced4176d16b0dd"}
Sep 11 12:15:03.774104 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.774020    2586 scope.go:117] "RemoveContainer" containerID="3ef1ee3438177fe67a27bf5a8023485ff55794998e2601900ed138ddc850962a"
Sep 11 12:15:03.775678 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.775656    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"
Sep 11 12:15:03.775775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.775685    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq" event={"ID":"f9913708-b6f6-4690-ac4a-fd2022bdfe36","Type":"ContainerDied","Data":"f0d2e55c6e05507208759e61a3276f88fb6e3eaa8b8e9323d420b1e1a5cbce1d"}
Sep 11 12:15:03.775775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.775742    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0d2e55c6e05507208759e61a3276f88fb6e3eaa8b8e9323d420b1e1a5cbce1d"
Sep 11 12:15:03.806000 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.805979    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"]
Sep 11 12:15:03.808553 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:03.808530    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-482cb0-29818815-s9dnn"]
Sep 11 12:15:04.762584 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:04.762532    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"]
Sep 11 12:15:04.785610 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:04.785584    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:04.824448 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:04.824418    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="360e8251-5c30-4d42-ac76-eed34e74b168" path="/var/lib/kubelet/pods/360e8251-5c30-4d42-ac76-eed34e74b168/volumes"
Sep 11 12:15:05.153870 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:05.153838    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:05.792537 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:05.792510    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"]
Sep 11 12:15:05.805629 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:05.805604    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:07.328103 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.328069    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:07.428144 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.428114    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:07.565026 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.564993    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:07.889323 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889285    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"]
Sep 11 12:15:07.889815 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889796    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="360e8251-5c30-4d42-ac76-eed34e74b168" containerName="cleanup"
Sep 11 12:15:07.889815 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889815    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="360e8251-5c30-4d42-ac76-eed34e74b168" containerName="cleanup"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889839    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0a8bf7-3ae9-4858-8891-2defc1ec8360" containerName="payload-processing"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889848    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0a8bf7-3ae9-4858-8891-2defc1ec8360" containerName="payload-processing"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889857    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="360e8251-5c30-4d42-ac76-eed34e74b168" containerName="cleanup"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889864    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="360e8251-5c30-4d42-ac76-eed34e74b168" containerName="cleanup"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889873    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26fd313e-217a-4b7e-a3be-0a6c0129d7af" containerName="payload-pre-processing"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889879    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="26fd313e-217a-4b7e-a3be-0a6c0129d7af" containerName="payload-pre-processing"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889885    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efd514b-7d02-4fdb-8170-4f0fd3f82f14" containerName="cleanup"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889890    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efd514b-7d02-4fdb-8170-4f0fd3f82f14" containerName="cleanup"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889904    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9913708-b6f6-4690-ac4a-fd2022bdfe36" containerName="cleanup"
Sep 11 12:15:07.889975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889909    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9913708-b6f6-4690-ac4a-fd2022bdfe36" containerName="cleanup"
Sep 11 12:15:07.890277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889982    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="360e8251-5c30-4d42-ac76-eed34e74b168" containerName="cleanup"
Sep 11 12:15:07.890277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.889992    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0a8bf7-3ae9-4858-8891-2defc1ec8360" containerName="payload-processing"
Sep 11 12:15:07.890277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.890002    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="360e8251-5c30-4d42-ac76-eed34e74b168" containerName="cleanup"
Sep 11 12:15:07.890277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.890012    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9913708-b6f6-4690-ac4a-fd2022bdfe36" containerName="cleanup"
Sep 11 12:15:07.890277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.890019    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8efd514b-7d02-4fdb-8170-4f0fd3f82f14" containerName="cleanup"
Sep 11 12:15:07.890277 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.890023    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="26fd313e-217a-4b7e-a3be-0a6c0129d7af" containerName="payload-pre-processing"
Sep 11 12:15:07.894283 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.894265    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"
Sep 11 12:15:07.897830 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.897808    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:15:07.897947 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.897807    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:15:07.902475 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:07.902444    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"]
Sep 11 12:15:08.026780 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:08.026744    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db6e790-2284-446a-bf98-a7f5d4d9fc4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-571fee-qrgqn\" (UID: \"0db6e790-2284-446a-bf98-a7f5d4d9fc4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"
Sep 11 12:15:08.041305 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:08.041280    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:08.127621 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:08.127594    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db6e790-2284-446a-bf98-a7f5d4d9fc4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-571fee-qrgqn\" (UID: \"0db6e790-2284-446a-bf98-a7f5d4d9fc4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"
Sep 11 12:15:08.128139 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:08.128121    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db6e790-2284-446a-bf98-a7f5d4d9fc4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-571fee-qrgqn\" (UID: \"0db6e790-2284-446a-bf98-a7f5d4d9fc4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"
Sep 11 12:15:08.207296 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:08.207239    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"
Sep 11 12:15:08.479402 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:08.479326    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"]
Sep 11 12:15:08.482784 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:15:08.482757    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db6e790_2284_446a_bf98_a7f5d4d9fc4a.slice/crio-1a268633f2c27d86df16342cebfa7ff810ae96e4099a7bb6247fe6981c020980 WatchSource:0}: Error finding container 1a268633f2c27d86df16342cebfa7ff810ae96e4099a7bb6247fe6981c020980: Status 404 returned error can't find the container with id 1a268633f2c27d86df16342cebfa7ff810ae96e4099a7bb6247fe6981c020980
Sep 11 12:15:08.806584 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:08.806555    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn" event={"ID":"0db6e790-2284-446a-bf98-a7f5d4d9fc4a","Type":"ContainerStarted","Data":"1a268633f2c27d86df16342cebfa7ff810ae96e4099a7bb6247fe6981c020980"}
Sep 11 12:15:09.812950 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:09.812919    2586 generic.go:356] "Generic (PLEG): container finished" podID="0db6e790-2284-446a-bf98-a7f5d4d9fc4a" containerID="23ece39c2868499448f9ee19a3049c623be47e1bf4bd448a352c237b8786665d" exitCode=0
Sep 11 12:15:09.813349 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:09.812979    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn" event={"ID":"0db6e790-2284-446a-bf98-a7f5d4d9fc4a","Type":"ContainerDied","Data":"23ece39c2868499448f9ee19a3049c623be47e1bf4bd448a352c237b8786665d"}
Sep 11 12:15:10.957159 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:10.957136    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"
Sep 11 12:15:11.051896 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:11.051867    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db6e790-2284-446a-bf98-a7f5d4d9fc4a-maas-api-service-ca\") pod \"0db6e790-2284-446a-bf98-a7f5d4d9fc4a\" (UID: \"0db6e790-2284-446a-bf98-a7f5d4d9fc4a\") "
Sep 11 12:15:11.052203 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:11.052180    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db6e790-2284-446a-bf98-a7f5d4d9fc4a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0db6e790-2284-446a-bf98-a7f5d4d9fc4a" (UID: "0db6e790-2284-446a-bf98-a7f5d4d9fc4a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:11.153560 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:11.153455    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0db6e790-2284-446a-bf98-a7f5d4d9fc4a-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:11.825652 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:11.825618    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn" event={"ID":"0db6e790-2284-446a-bf98-a7f5d4d9fc4a","Type":"ContainerDied","Data":"1a268633f2c27d86df16342cebfa7ff810ae96e4099a7bb6247fe6981c020980"}
Sep 11 12:15:11.825807 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:11.825672    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a268633f2c27d86df16342cebfa7ff810ae96e4099a7bb6247fe6981c020980"
Sep 11 12:15:11.825807 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:11.825625    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"
Sep 11 12:15:18.295812 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:18.295769    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:15:19.194590 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:19.194553    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:19.782954 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:19.782916    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms" podUID="630f476f-3617-4012-b482-647337880f45" containerName="main" containerID="cri-o://13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4" gracePeriod=45
Sep 11 12:15:20.060047 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.060026    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:15:20.126008 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.125985    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-tmp-dir\") pod \"630f476f-3617-4012-b482-647337880f45\" (UID: \"630f476f-3617-4012-b482-647337880f45\") "
Sep 11 12:15:20.126155 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126041    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-dshm\") pod \"630f476f-3617-4012-b482-647337880f45\" (UID: \"630f476f-3617-4012-b482-647337880f45\") "
Sep 11 12:15:20.126155 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126069    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-model-cache\") pod \"630f476f-3617-4012-b482-647337880f45\" (UID: \"630f476f-3617-4012-b482-647337880f45\") "
Sep 11 12:15:20.126155 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126113    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh7pn\" (UniqueName: \"kubernetes.io/projected/630f476f-3617-4012-b482-647337880f45-kube-api-access-fh7pn\") pod \"630f476f-3617-4012-b482-647337880f45\" (UID: \"630f476f-3617-4012-b482-647337880f45\") "
Sep 11 12:15:20.126327 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126159    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630f476f-3617-4012-b482-647337880f45-tls-certs\") pod \"630f476f-3617-4012-b482-647337880f45\" (UID: \"630f476f-3617-4012-b482-647337880f45\") "
Sep 11 12:15:20.126327 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126205    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-home\") pod \"630f476f-3617-4012-b482-647337880f45\" (UID: \"630f476f-3617-4012-b482-647337880f45\") "
Sep 11 12:15:20.126327 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126271    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "630f476f-3617-4012-b482-647337880f45" (UID: "630f476f-3617-4012-b482-647337880f45"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:20.126456 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126328    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-model-cache" (OuterVolumeSpecName: "model-cache") pod "630f476f-3617-4012-b482-647337880f45" (UID: "630f476f-3617-4012-b482-647337880f45"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:20.126566 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126536    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-home" (OuterVolumeSpecName: "home") pod "630f476f-3617-4012-b482-647337880f45" (UID: "630f476f-3617-4012-b482-647337880f45"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:20.126566 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126554    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:20.126722 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.126580    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:20.128376 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.128356    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/630f476f-3617-4012-b482-647337880f45-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "630f476f-3617-4012-b482-647337880f45" (UID: "630f476f-3617-4012-b482-647337880f45"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:20.128765 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.128741    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/630f476f-3617-4012-b482-647337880f45-kube-api-access-fh7pn" (OuterVolumeSpecName: "kube-api-access-fh7pn") pod "630f476f-3617-4012-b482-647337880f45" (UID: "630f476f-3617-4012-b482-647337880f45"). InnerVolumeSpecName "kube-api-access-fh7pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:20.128765 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.128752    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-dshm" (OuterVolumeSpecName: "dshm") pod "630f476f-3617-4012-b482-647337880f45" (UID: "630f476f-3617-4012-b482-647337880f45"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:20.227660 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.227639    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:20.227660 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.227658    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh7pn\" (UniqueName: \"kubernetes.io/projected/630f476f-3617-4012-b482-647337880f45-kube-api-access-fh7pn\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:20.227778 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.227668    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630f476f-3617-4012-b482-647337880f45-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:20.227778 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.227676    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630f476f-3617-4012-b482-647337880f45-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:20.811885 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.811847    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6" podUID="29cb4fa1-a19f-4764-a486-f8c31370edf4" containerName="main" containerID="cri-o://4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1" gracePeriod=45
Sep 11 12:15:20.871084 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.871054    2586 generic.go:356] "Generic (PLEG): container finished" podID="630f476f-3617-4012-b482-647337880f45" containerID="13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4" exitCode=0
Sep 11 12:15:20.871185 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.871101    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"
Sep 11 12:15:20.871185 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.871148    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms" event={"ID":"630f476f-3617-4012-b482-647337880f45","Type":"ContainerDied","Data":"13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4"}
Sep 11 12:15:20.871262 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.871200    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms" event={"ID":"630f476f-3617-4012-b482-647337880f45","Type":"ContainerDied","Data":"2df904ddfe1667890848b904ea3c6859cba66070d45d7940331f566b1b37f8b1"}
Sep 11 12:15:20.871262 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.871223    2586 scope.go:117] "RemoveContainer" containerID="13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4"
Sep 11 12:15:20.891830 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.891803    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"]
Sep 11 12:15:20.897446 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.897424    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/sub-test-model-88968d-kserve-597fd48d57-9ptms"]
Sep 11 12:15:20.923228 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.923207    2586 scope.go:117] "RemoveContainer" containerID="13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4"
Sep 11 12:15:20.923550 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:20.923530    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4\": container with ID starting with 13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4 not found: ID does not exist" containerID="13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4"
Sep 11 12:15:20.923626 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:20.923564    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4"} err="failed to get container status \"13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4\": rpc error: code = NotFound desc = could not find container \"13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4\": container with ID starting with 13e4e906f67c72e48644bdc56753634cb3c631d5c255ce1ade1c198a54fddce4 not found: ID does not exist"
Sep 11 12:15:21.070880 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.070825    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:15:21.236305 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236276    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-dshm\") pod \"29cb4fa1-a19f-4764-a486-f8c31370edf4\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") "
Sep 11 12:15:21.236416 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236313    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-tmp-dir\") pod \"29cb4fa1-a19f-4764-a486-f8c31370edf4\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") "
Sep 11 12:15:21.236416 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236339    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-home\") pod \"29cb4fa1-a19f-4764-a486-f8c31370edf4\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") "
Sep 11 12:15:21.236416 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236382    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkkdn\" (UniqueName: \"kubernetes.io/projected/29cb4fa1-a19f-4764-a486-f8c31370edf4-kube-api-access-vkkdn\") pod \"29cb4fa1-a19f-4764-a486-f8c31370edf4\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") "
Sep 11 12:15:21.236416 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236407    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29cb4fa1-a19f-4764-a486-f8c31370edf4-tls-certs\") pod \"29cb4fa1-a19f-4764-a486-f8c31370edf4\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") "
Sep 11 12:15:21.236657 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236446    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-model-cache\") pod \"29cb4fa1-a19f-4764-a486-f8c31370edf4\" (UID: \"29cb4fa1-a19f-4764-a486-f8c31370edf4\") "
Sep 11 12:15:21.236657 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236592    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29cb4fa1-a19f-4764-a486-f8c31370edf4" (UID: "29cb4fa1-a19f-4764-a486-f8c31370edf4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:21.236766 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236697    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-home" (OuterVolumeSpecName: "home") pod "29cb4fa1-a19f-4764-a486-f8c31370edf4" (UID: "29cb4fa1-a19f-4764-a486-f8c31370edf4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:21.236836 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236797    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:21.236836 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236809    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-model-cache" (OuterVolumeSpecName: "model-cache") pod "29cb4fa1-a19f-4764-a486-f8c31370edf4" (UID: "29cb4fa1-a19f-4764-a486-f8c31370edf4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:21.236911 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.236817    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:21.238641 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.238619    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-dshm" (OuterVolumeSpecName: "dshm") pod "29cb4fa1-a19f-4764-a486-f8c31370edf4" (UID: "29cb4fa1-a19f-4764-a486-f8c31370edf4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:15:21.238784 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.238770    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29cb4fa1-a19f-4764-a486-f8c31370edf4-kube-api-access-vkkdn" (OuterVolumeSpecName: "kube-api-access-vkkdn") pod "29cb4fa1-a19f-4764-a486-f8c31370edf4" (UID: "29cb4fa1-a19f-4764-a486-f8c31370edf4"). InnerVolumeSpecName "kube-api-access-vkkdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:21.238903 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.238884    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29cb4fa1-a19f-4764-a486-f8c31370edf4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29cb4fa1-a19f-4764-a486-f8c31370edf4" (UID: "29cb4fa1-a19f-4764-a486-f8c31370edf4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:21.338009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.337955    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkkdn\" (UniqueName: \"kubernetes.io/projected/29cb4fa1-a19f-4764-a486-f8c31370edf4-kube-api-access-vkkdn\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:21.338009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.337977    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29cb4fa1-a19f-4764-a486-f8c31370edf4-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:21.338009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.337986    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:21.338009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.337995    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29cb4fa1-a19f-4764-a486-f8c31370edf4-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:21.878703 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.878666    2586 generic.go:356] "Generic (PLEG): container finished" podID="29cb4fa1-a19f-4764-a486-f8c31370edf4" containerID="4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1" exitCode=0
Sep 11 12:15:21.879103 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.878713    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"
Sep 11 12:15:21.879103 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.878739    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6" event={"ID":"29cb4fa1-a19f-4764-a486-f8c31370edf4","Type":"ContainerDied","Data":"4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1"}
Sep 11 12:15:21.879103 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.878770    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6" event={"ID":"29cb4fa1-a19f-4764-a486-f8c31370edf4","Type":"ContainerDied","Data":"7a7e97f5160bec3afab4d1ff8d98b13016af859c1d402318cbbb7b0682ed87bf"}
Sep 11 12:15:21.879103 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.878789    2586 scope.go:117] "RemoveContainer" containerID="4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1"
Sep 11 12:15:21.887911 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.887895    2586 scope.go:117] "RemoveContainer" containerID="4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1"
Sep 11 12:15:21.888166 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:21.888147    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1\": container with ID starting with 4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1 not found: ID does not exist" containerID="4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1"
Sep 11 12:15:21.888220 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.888173    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1"} err="failed to get container status \"4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1\": rpc error: code = NotFound desc = could not find container \"4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1\": container with ID starting with 4375bb7faca2af5922e9c973f66152a3bc61c021c6bd5449da27570c3ed0e3a1 not found: ID does not exist"
Sep 11 12:15:21.900377 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.900357    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"]
Sep 11 12:15:21.905931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:21.905909    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/sub-test-model-4d3f83-kserve-6d5c6b556c-lqvw6"]
Sep 11 12:15:22.823770 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:22.823724    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29cb4fa1-a19f-4764-a486-f8c31370edf4" path="/var/lib/kubelet/pods/29cb4fa1-a19f-4764-a486-f8c31370edf4/volumes"
Sep 11 12:15:22.824223 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:22.824199    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="630f476f-3617-4012-b482-647337880f45" path="/var/lib/kubelet/pods/630f476f-3617-4012-b482-647337880f45/volumes"
Sep 11 12:15:23.536164 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.536135    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"]
Sep 11 12:15:23.538915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.538881    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:23.539340 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.539322    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-571fee-29818815-gfpwq"]
Sep 11 12:15:23.638965 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.638938    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:23.738756 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.738729    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"]
Sep 11 12:15:23.741737 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.741705    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:23.743224 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.743205    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-571fee-qrgqn"]
Sep 11 12:15:23.839226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:23.839198    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:24.097295 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:24.097210    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:24.823960 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:24.823925    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0db6e790-2284-446a-bf98-a7f5d4d9fc4a" path="/var/lib/kubelet/pods/0db6e790-2284-446a-bf98-a7f5d4d9fc4a/volumes"
Sep 11 12:15:24.824327 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:24.824202    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9913708-b6f6-4690-ac4a-fd2022bdfe36" path="/var/lib/kubelet/pods/f9913708-b6f6-4690-ac4a-fd2022bdfe36/volumes"
Sep 11 12:15:28.303151 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:28.303114    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:15:29.834526 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:29.834468    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:30.046899 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.046866    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:30.118064 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.117984    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:30.425866 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.425794    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"]
Sep 11 12:15:30.426169 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426156    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="630f476f-3617-4012-b482-647337880f45" containerName="main"
Sep 11 12:15:30.426221 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426171    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="630f476f-3617-4012-b482-647337880f45" containerName="main"
Sep 11 12:15:30.426221 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426181    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db6e790-2284-446a-bf98-a7f5d4d9fc4a" containerName="revoke-keys"
Sep 11 12:15:30.426221 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426186    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db6e790-2284-446a-bf98-a7f5d4d9fc4a" containerName="revoke-keys"
Sep 11 12:15:30.426221 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426199    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29cb4fa1-a19f-4764-a486-f8c31370edf4" containerName="main"
Sep 11 12:15:30.426221 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426205    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="29cb4fa1-a19f-4764-a486-f8c31370edf4" containerName="main"
Sep 11 12:15:30.426376 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426273    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="29cb4fa1-a19f-4764-a486-f8c31370edf4" containerName="main"
Sep 11 12:15:30.426376 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426282    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db6e790-2284-446a-bf98-a7f5d4d9fc4a" containerName="revoke-keys"
Sep 11 12:15:30.426376 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.426291    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="630f476f-3617-4012-b482-647337880f45" containerName="main"
Sep 11 12:15:30.430844 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.430827    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"
Sep 11 12:15:30.434611 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.434582    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:15:30.434755 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.434628    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:15:30.436040 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.436017    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"]
Sep 11 12:15:30.511096 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.511071    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83ea0871-d3a3-4006-a7f7-03c77a6f7f47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr\" (UID: \"83ea0871-d3a3-4006-a7f7-03c77a6f7f47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"
Sep 11 12:15:30.611440 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.611415    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83ea0871-d3a3-4006-a7f7-03c77a6f7f47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr\" (UID: \"83ea0871-d3a3-4006-a7f7-03c77a6f7f47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"
Sep 11 12:15:30.611990 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.611969    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83ea0871-d3a3-4006-a7f7-03c77a6f7f47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr\" (UID: \"83ea0871-d3a3-4006-a7f7-03c77a6f7f47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"
Sep 11 12:15:30.684975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.684916    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:30.743194 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:30.743171    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"
Sep 11 12:15:31.017568 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:31.017549    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"]
Sep 11 12:15:31.019395 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:15:31.019370    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ea0871_d3a3_4006_a7f7_03c77a6f7f47.slice/crio-a6b5fc79031aff60686c7b63b0ac4e6480e0a459b65a6af9e1508b23e7283629 WatchSource:0}: Error finding container a6b5fc79031aff60686c7b63b0ac4e6480e0a459b65a6af9e1508b23e7283629: Status 404 returned error can't find the container with id a6b5fc79031aff60686c7b63b0ac4e6480e0a459b65a6af9e1508b23e7283629
Sep 11 12:15:31.939600 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:31.939526    2586 generic.go:356] "Generic (PLEG): container finished" podID="83ea0871-d3a3-4006-a7f7-03c77a6f7f47" containerID="ee733455420944e0bf9fd3b482ff4d97a9cdb7dffd40ce8990ace8acca4e5ab3" exitCode=0
Sep 11 12:15:31.939729 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:31.939611    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr" event={"ID":"83ea0871-d3a3-4006-a7f7-03c77a6f7f47","Type":"ContainerDied","Data":"ee733455420944e0bf9fd3b482ff4d97a9cdb7dffd40ce8990ace8acca4e5ab3"}
Sep 11 12:15:31.939729 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:31.939681    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr" event={"ID":"83ea0871-d3a3-4006-a7f7-03c77a6f7f47","Type":"ContainerStarted","Data":"a6b5fc79031aff60686c7b63b0ac4e6480e0a459b65a6af9e1508b23e7283629"}
Sep 11 12:15:32.361115 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.361085    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"]
Sep 11 12:15:32.364534 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.364516    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.368555 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.368532    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"openshift-service-ca.crt\""
Sep 11 12:15:32.368555 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.368546    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"kube-root-ca.crt\""
Sep 11 12:15:32.368725 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.368642    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"rate-test-model-88968d-kserve-self-signed-certs\""
Sep 11 12:15:32.370464 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.370447    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-88968d\"/\"default-dockercfg-sqc5s\""
Sep 11 12:15:32.388184 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.388162    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"]
Sep 11 12:15:32.524815 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.524788    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29ba867-3bd2-49a3-b740-28bda9659786-tls-certs\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.524919 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.524823    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5sgg\" (UniqueName: \"kubernetes.io/projected/b29ba867-3bd2-49a3-b740-28bda9659786-kube-api-access-t5sgg\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.524919 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.524861    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-home\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.524919 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.524916    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-tmp-dir\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.525030 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.524946    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-dshm\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.525030 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.524986    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-model-cache\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.615680 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.615619    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:32.625929 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.625904    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-model-cache\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626034 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.625982    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29ba867-3bd2-49a3-b740-28bda9659786-tls-certs\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626101 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.626020    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5sgg\" (UniqueName: \"kubernetes.io/projected/b29ba867-3bd2-49a3-b740-28bda9659786-kube-api-access-t5sgg\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626101 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.626081    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-home\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626212 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.626141    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-tmp-dir\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626212 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.626191    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-dshm\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626335 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.626261    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-model-cache\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626440 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.626421    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-home\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.626550 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.626532    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-tmp-dir\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.628412 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.628392    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-dshm\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.628580 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.628565    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29ba867-3bd2-49a3-b740-28bda9659786-tls-certs\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.634640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.634610    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5sgg\" (UniqueName: \"kubernetes.io/projected/b29ba867-3bd2-49a3-b740-28bda9659786-kube-api-access-t5sgg\") pod \"rate-test-model-88968d-kserve-78849459c5-9zhbx\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") " pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.674214 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.674194    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:32.945927 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:32.945855    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"]
Sep 11 12:15:32.948242 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:15:32.948220    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb29ba867_3bd2_49a3_b740_28bda9659786.slice/crio-96c0111bbf1530aa7ff276717300faa888cf522a295e9126b55b071a1b33a490 WatchSource:0}: Error finding container 96c0111bbf1530aa7ff276717300faa888cf522a295e9126b55b071a1b33a490: Status 404 returned error can't find the container with id 96c0111bbf1530aa7ff276717300faa888cf522a295e9126b55b071a1b33a490
Sep 11 12:15:33.073678 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.073652    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"
Sep 11 12:15:33.232335 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.232255    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83ea0871-d3a3-4006-a7f7-03c77a6f7f47-maas-api-service-ca\") pod \"83ea0871-d3a3-4006-a7f7-03c77a6f7f47\" (UID: \"83ea0871-d3a3-4006-a7f7-03c77a6f7f47\") "
Sep 11 12:15:33.232668 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.232639    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83ea0871-d3a3-4006-a7f7-03c77a6f7f47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "83ea0871-d3a3-4006-a7f7-03c77a6f7f47" (UID: "83ea0871-d3a3-4006-a7f7-03c77a6f7f47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:33.333719 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.333694    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83ea0871-d3a3-4006-a7f7-03c77a6f7f47-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:33.951252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.951190    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"
Sep 11 12:15:33.951252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.951198    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr" event={"ID":"83ea0871-d3a3-4006-a7f7-03c77a6f7f47","Type":"ContainerDied","Data":"a6b5fc79031aff60686c7b63b0ac4e6480e0a459b65a6af9e1508b23e7283629"}
Sep 11 12:15:33.951252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.951246    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6b5fc79031aff60686c7b63b0ac4e6480e0a459b65a6af9e1508b23e7283629"
Sep 11 12:15:33.953044 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.953021    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx" event={"ID":"b29ba867-3bd2-49a3-b740-28bda9659786","Type":"ContainerStarted","Data":"d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa"}
Sep 11 12:15:33.953145 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.953060    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx" event={"ID":"b29ba867-3bd2-49a3-b740-28bda9659786","Type":"ContainerStarted","Data":"96c0111bbf1530aa7ff276717300faa888cf522a295e9126b55b071a1b33a490"}
Sep 11 12:15:33.973143 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:33.973100    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx" podStartSLOduration=1.973087374 podStartE2EDuration="1.973087374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:15:33.970920084 +0000 UTC m=+1427.681591424" watchObservedRunningTime="2026-09-11 12:15:33.973087374 +0000 UTC m=+1427.683758711"
Sep 11 12:15:38.201853 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:38.201767    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:15:40.940716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:40.940681    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:40.970073 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:40.970043    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"]
Sep 11 12:15:40.970570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:40.970539    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c" podUID="c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" containerName="payload-processing" containerID="cri-o://14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668" gracePeriod=30
Sep 11 12:15:40.979601 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:40.979575    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"]
Sep 11 12:15:40.979827 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:40.979786    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9" podUID="8f596b9d-641e-483c-9dd5-9d8da72ad92f" containerName="payload-pre-processing" containerID="cri-o://d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd" gracePeriod=30
Sep 11 12:15:41.015028 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.015000    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:41.147073 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.147013    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"]
Sep 11 12:15:41.149030 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.149006    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c000cf-fh4sr"]
Sep 11 12:15:41.153076 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.153049    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:41.223727 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.223700    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:41.242993 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.242970    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:15:41.261890 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.261872    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:15:41.308495 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.308467    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-plugins-config-volume\") pod \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") "
Sep 11 12:15:41.308634 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.308558    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g4ct\" (UniqueName: \"kubernetes.io/projected/8f596b9d-641e-483c-9dd5-9d8da72ad92f-kube-api-access-4g4ct\") pod \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") "
Sep 11 12:15:41.308634 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.308578    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbh8g\" (UniqueName: \"kubernetes.io/projected/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-kube-api-access-qbh8g\") pod \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\" (UID: \"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6\") "
Sep 11 12:15:41.308634 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.308631    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f596b9d-641e-483c-9dd5-9d8da72ad92f-plugins-config-volume\") pod \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\" (UID: \"8f596b9d-641e-483c-9dd5-9d8da72ad92f\") "
Sep 11 12:15:41.308989 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.308850    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" (UID: "c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:41.309103 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.309008    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f596b9d-641e-483c-9dd5-9d8da72ad92f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f596b9d-641e-483c-9dd5-9d8da72ad92f" (UID: "8f596b9d-641e-483c-9dd5-9d8da72ad92f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:41.310711 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.310688    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f596b9d-641e-483c-9dd5-9d8da72ad92f-kube-api-access-4g4ct" (OuterVolumeSpecName: "kube-api-access-4g4ct") pod "8f596b9d-641e-483c-9dd5-9d8da72ad92f" (UID: "8f596b9d-641e-483c-9dd5-9d8da72ad92f"). InnerVolumeSpecName "kube-api-access-4g4ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:41.310799 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.310737    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-kube-api-access-qbh8g" (OuterVolumeSpecName: "kube-api-access-qbh8g") pod "c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" (UID: "c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6"). InnerVolumeSpecName "kube-api-access-qbh8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:41.409255 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.409233    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g4ct\" (UniqueName: \"kubernetes.io/projected/8f596b9d-641e-483c-9dd5-9d8da72ad92f-kube-api-access-4g4ct\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.409255 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.409253    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbh8g\" (UniqueName: \"kubernetes.io/projected/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-kube-api-access-qbh8g\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.409451 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.409262    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f596b9d-641e-483c-9dd5-9d8da72ad92f-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.409451 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.409270    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.494367 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.494301    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:41.997460 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.997426    2586 generic.go:356] "Generic (PLEG): container finished" podID="c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" containerID="14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668" exitCode=0
Sep 11 12:15:41.997895 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.997512    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"
Sep 11 12:15:41.997895 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.997512    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c" event={"ID":"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6","Type":"ContainerDied","Data":"14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668"}
Sep 11 12:15:41.997895 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.997567    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c" event={"ID":"c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6","Type":"ContainerDied","Data":"5eece2654ea8a08ea26fc723c6149537847c47468ab44153aa5080104e281756"}
Sep 11 12:15:41.997895 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.997590    2586 scope.go:117] "RemoveContainer" containerID="14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668"
Sep 11 12:15:41.998889 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.998844    2586 generic.go:356] "Generic (PLEG): container finished" podID="8f596b9d-641e-483c-9dd5-9d8da72ad92f" containerID="d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd" exitCode=0
Sep 11 12:15:41.998889 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.998875    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9" event={"ID":"8f596b9d-641e-483c-9dd5-9d8da72ad92f","Type":"ContainerDied","Data":"d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd"}
Sep 11 12:15:41.999025 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.998906    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9" event={"ID":"8f596b9d-641e-483c-9dd5-9d8da72ad92f","Type":"ContainerDied","Data":"aef1cda942299c84e98cc1f3e3994c6d761d28b1814393308fd1180e7b33a977"}
Sep 11 12:15:41.999025 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:41.998909    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"
Sep 11 12:15:42.007775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.007760    2586 scope.go:117] "RemoveContainer" containerID="14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668"
Sep 11 12:15:42.008028 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:42.008009    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668\": container with ID starting with 14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668 not found: ID does not exist" containerID="14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668"
Sep 11 12:15:42.008118 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.008035    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668"} err="failed to get container status \"14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668\": rpc error: code = NotFound desc = could not find container \"14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668\": container with ID starting with 14fbd07db447af1b96402057fa440cad9afab9691dedf6eca422031716007668 not found: ID does not exist"
Sep 11 12:15:42.008118 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.008051    2586 scope.go:117] "RemoveContainer" containerID="d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd"
Sep 11 12:15:42.015996 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.015980    2586 scope.go:117] "RemoveContainer" containerID="d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd"
Sep 11 12:15:42.016243 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:42.016226    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd\": container with ID starting with d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd not found: ID does not exist" containerID="d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd"
Sep 11 12:15:42.016298 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.016249    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd"} err="failed to get container status \"d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd\": rpc error: code = NotFound desc = could not find container \"d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd\": container with ID starting with d6c234534c24ee325a5d83e2eb2564d6e63a513e81efac19c632ed1896baccdd not found: ID does not exist"
Sep 11 12:15:42.023348 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.023326    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"]
Sep 11 12:15:42.025577 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.025556    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c000cf-6dc66b5d9-mg77c"]
Sep 11 12:15:42.036896 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.036872    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"]
Sep 11 12:15:42.038843 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.038825    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c000cf-8698d95dcd-gvdr9"]
Sep 11 12:15:42.675138 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.675106    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:42.675296 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.675159    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:42.688031 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.688005    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:42.824137 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.824106    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83ea0871-d3a3-4006-a7f7-03c77a6f7f47" path="/var/lib/kubelet/pods/83ea0871-d3a3-4006-a7f7-03c77a6f7f47/volumes"
Sep 11 12:15:42.824452 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.824439    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f596b9d-641e-483c-9dd5-9d8da72ad92f" path="/var/lib/kubelet/pods/8f596b9d-641e-483c-9dd5-9d8da72ad92f/volumes"
Sep 11 12:15:42.824770 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:42.824758    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" path="/var/lib/kubelet/pods/c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6/volumes"
Sep 11 12:15:43.017529 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:43.017432    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:15:47.005226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.005196    2586 scope.go:117] "RemoveContainer" containerID="76e3d4584c379be1debb8834b34685883dd747a639e329628e93fbc487f0a98f"
Sep 11 12:15:47.388405 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.388375    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:47.490121 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.490088    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:47.630743 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.630713    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:47.985999 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.985968    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"]
Sep 11 12:15:47.986378 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986365    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f596b9d-641e-483c-9dd5-9d8da72ad92f" containerName="payload-pre-processing"
Sep 11 12:15:47.986438 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986379    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f596b9d-641e-483c-9dd5-9d8da72ad92f" containerName="payload-pre-processing"
Sep 11 12:15:47.986438 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986387    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" containerName="payload-processing"
Sep 11 12:15:47.986438 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986393    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" containerName="payload-processing"
Sep 11 12:15:47.986438 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986398    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83ea0871-d3a3-4006-a7f7-03c77a6f7f47" containerName="revoke-keys"
Sep 11 12:15:47.986438 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986403    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="83ea0871-d3a3-4006-a7f7-03c77a6f7f47" containerName="revoke-keys"
Sep 11 12:15:47.986655 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986473    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9bdf33e-93cd-4804-a2fc-b4005c6ae3b6" containerName="payload-processing"
Sep 11 12:15:47.986655 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986499    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="83ea0871-d3a3-4006-a7f7-03c77a6f7f47" containerName="revoke-keys"
Sep 11 12:15:47.986655 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.986508    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f596b9d-641e-483c-9dd5-9d8da72ad92f" containerName="payload-pre-processing"
Sep 11 12:15:47.991206 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.991187    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"
Sep 11 12:15:47.994364 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.994342    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:15:47.994530 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.994400    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:15:47.998602 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:47.998577    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"]
Sep 11 12:15:48.061534 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:48.061506    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dee54-1ac6-40dc-b762-9e74cb635aaf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln\" (UID: \"6a9dee54-1ac6-40dc-b762-9e74cb635aaf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"
Sep 11 12:15:48.085006 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:48.084982    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:48.162788 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:48.162765    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dee54-1ac6-40dc-b762-9e74cb635aaf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln\" (UID: \"6a9dee54-1ac6-40dc-b762-9e74cb635aaf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"
Sep 11 12:15:48.163344 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:48.163327    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dee54-1ac6-40dc-b762-9e74cb635aaf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln\" (UID: \"6a9dee54-1ac6-40dc-b762-9e74cb635aaf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"
Sep 11 12:15:48.304193 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:48.304167    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"
Sep 11 12:15:48.578642 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:48.578548    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"]
Sep 11 12:15:48.580839 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:15:48.580812    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9dee54_1ac6_40dc_b762_9e74cb635aaf.slice/crio-c04ef482b5f82d6d4d767512b5f7c778f68829d0efdd857b211f1fcec1de3405 WatchSource:0}: Error finding container c04ef482b5f82d6d4d767512b5f7c778f68829d0efdd857b211f1fcec1de3405: Status 404 returned error can't find the container with id c04ef482b5f82d6d4d767512b5f7c778f68829d0efdd857b211f1fcec1de3405
Sep 11 12:15:49.041479 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:49.041449    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln" event={"ID":"6a9dee54-1ac6-40dc-b762-9e74cb635aaf","Type":"ContainerStarted","Data":"f7c318cc523b9717ba5c440dff768c98aadce83f8955bbf31b15560508c79ee1"}
Sep 11 12:15:49.041628 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:49.041520    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln" event={"ID":"6a9dee54-1ac6-40dc-b762-9e74cb635aaf","Type":"ContainerStarted","Data":"c04ef482b5f82d6d4d767512b5f7c778f68829d0efdd857b211f1fcec1de3405"}
Sep 11 12:15:49.060858 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:49.060815    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln" podStartSLOduration=2.060800794 podStartE2EDuration="2.060800794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:15:49.059570992 +0000 UTC m=+1442.770242353" watchObservedRunningTime="2026-09-11 12:15:49.060800794 +0000 UTC m=+1442.771472123"
Sep 11 12:15:50.048525 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:50.048471    2586 generic.go:356] "Generic (PLEG): container finished" podID="6a9dee54-1ac6-40dc-b762-9e74cb635aaf" containerID="f7c318cc523b9717ba5c440dff768c98aadce83f8955bbf31b15560508c79ee1" exitCode=0
Sep 11 12:15:50.048880 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:50.048556    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln" event={"ID":"6a9dee54-1ac6-40dc-b762-9e74cb635aaf","Type":"ContainerDied","Data":"f7c318cc523b9717ba5c440dff768c98aadce83f8955bbf31b15560508c79ee1"}
Sep 11 12:15:50.397876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:50.397799    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:15:51.187412 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:51.187388    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"
Sep 11 12:15:51.294295 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:51.294271    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dee54-1ac6-40dc-b762-9e74cb635aaf-maas-api-service-ca\") pod \"6a9dee54-1ac6-40dc-b762-9e74cb635aaf\" (UID: \"6a9dee54-1ac6-40dc-b762-9e74cb635aaf\") "
Sep 11 12:15:51.294608 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:51.294589    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9dee54-1ac6-40dc-b762-9e74cb635aaf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a9dee54-1ac6-40dc-b762-9e74cb635aaf" (UID: "6a9dee54-1ac6-40dc-b762-9e74cb635aaf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:51.395122 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:51.395067    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dee54-1ac6-40dc-b762-9e74cb635aaf-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:52.060985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:52.060954    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"
Sep 11 12:15:52.060985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:52.060969    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln" event={"ID":"6a9dee54-1ac6-40dc-b762-9e74cb635aaf","Type":"ContainerDied","Data":"c04ef482b5f82d6d4d767512b5f7c778f68829d0efdd857b211f1fcec1de3405"}
Sep 11 12:15:52.061207 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:52.061015    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c04ef482b5f82d6d4d767512b5f7c778f68829d0efdd857b211f1fcec1de3405"
Sep 11 12:15:58.468422 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.468383    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:58.476521 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.476494    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"]
Sep 11 12:15:58.479271 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.479245    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-50e434-29818815-mffz8"]
Sep 11 12:15:58.498716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.498693    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"]
Sep 11 12:15:58.498889 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.498867    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk" podUID="35461298-3ebf-41bd-a4b1-ff7e2890e07c" containerName="payload-processing" containerID="cri-o://a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f" gracePeriod=30
Sep 11 12:15:58.503772 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.503747    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"]
Sep 11 12:15:58.503954 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.503919    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l" podUID="a2c95706-a2fb-4184-8dad-0631980ad76d" containerName="payload-pre-processing" containerID="cri-o://b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146" gracePeriod=30
Sep 11 12:15:58.539710 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.539685    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:58.602035 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.602008    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:58.703256 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.703208    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"]
Sep 11 12:15:58.705158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.705022    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:58.705400 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.705380    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-50e434-jq2ln"]
Sep 11 12:15:58.778565 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.778544    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:15:58.781189 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.781169    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:15:58.799467 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.799442    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:58.824098 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.824070    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a9dee54-1ac6-40dc-b762-9e74cb635aaf" path="/var/lib/kubelet/pods/6a9dee54-1ac6-40dc-b762-9e74cb635aaf/volumes"
Sep 11 12:15:58.824524 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.824501    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8efd514b-7d02-4fdb-8170-4f0fd3f82f14" path="/var/lib/kubelet/pods/8efd514b-7d02-4fdb-8170-4f0fd3f82f14/volumes"
Sep 11 12:15:58.852698 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.852675    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c95706-a2fb-4184-8dad-0631980ad76d-plugins-config-volume\") pod \"a2c95706-a2fb-4184-8dad-0631980ad76d\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") "
Sep 11 12:15:58.852779 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.852729    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwbfk\" (UniqueName: \"kubernetes.io/projected/35461298-3ebf-41bd-a4b1-ff7e2890e07c-kube-api-access-jwbfk\") pod \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") "
Sep 11 12:15:58.852779 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.852746    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng7p7\" (UniqueName: \"kubernetes.io/projected/a2c95706-a2fb-4184-8dad-0631980ad76d-kube-api-access-ng7p7\") pod \"a2c95706-a2fb-4184-8dad-0631980ad76d\" (UID: \"a2c95706-a2fb-4184-8dad-0631980ad76d\") "
Sep 11 12:15:58.852779 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.852772    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35461298-3ebf-41bd-a4b1-ff7e2890e07c-plugins-config-volume\") pod \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\" (UID: \"35461298-3ebf-41bd-a4b1-ff7e2890e07c\") "
Sep 11 12:15:58.853040 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.853011    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c95706-a2fb-4184-8dad-0631980ad76d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2c95706-a2fb-4184-8dad-0631980ad76d" (UID: "a2c95706-a2fb-4184-8dad-0631980ad76d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:58.853209 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.853187    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35461298-3ebf-41bd-a4b1-ff7e2890e07c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35461298-3ebf-41bd-a4b1-ff7e2890e07c" (UID: "35461298-3ebf-41bd-a4b1-ff7e2890e07c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:58.854878 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.854860    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c95706-a2fb-4184-8dad-0631980ad76d-kube-api-access-ng7p7" (OuterVolumeSpecName: "kube-api-access-ng7p7") pod "a2c95706-a2fb-4184-8dad-0631980ad76d" (UID: "a2c95706-a2fb-4184-8dad-0631980ad76d"). InnerVolumeSpecName "kube-api-access-ng7p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:58.854944 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.854917    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35461298-3ebf-41bd-a4b1-ff7e2890e07c-kube-api-access-jwbfk" (OuterVolumeSpecName: "kube-api-access-jwbfk") pod "35461298-3ebf-41bd-a4b1-ff7e2890e07c" (UID: "35461298-3ebf-41bd-a4b1-ff7e2890e07c"). InnerVolumeSpecName "kube-api-access-jwbfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:58.859086 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.859065    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:58.953836 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.953812    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c95706-a2fb-4184-8dad-0631980ad76d-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:58.953836 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.953831    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwbfk\" (UniqueName: \"kubernetes.io/projected/35461298-3ebf-41bd-a4b1-ff7e2890e07c-kube-api-access-jwbfk\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:58.953939 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.953843    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng7p7\" (UniqueName: \"kubernetes.io/projected/a2c95706-a2fb-4184-8dad-0631980ad76d-kube-api-access-ng7p7\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:58.953939 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:58.953851    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35461298-3ebf-41bd-a4b1-ff7e2890e07c-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:15:59.070386 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.070350    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:15:59.101739 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.101706    2586 generic.go:356] "Generic (PLEG): container finished" podID="35461298-3ebf-41bd-a4b1-ff7e2890e07c" containerID="a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f" exitCode=0
Sep 11 12:15:59.101902 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.101881    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk" event={"ID":"35461298-3ebf-41bd-a4b1-ff7e2890e07c","Type":"ContainerDied","Data":"a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f"}
Sep 11 12:15:59.101983 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.101968    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk" event={"ID":"35461298-3ebf-41bd-a4b1-ff7e2890e07c","Type":"ContainerDied","Data":"9a3fcbf5ecb7b55f3df02e9da56fd80fa9d64f4e8d6133761c106ca4b80220fd"}
Sep 11 12:15:59.102034 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.102013    2586 scope.go:117] "RemoveContainer" containerID="a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f"
Sep 11 12:15:59.102394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.102214    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"
Sep 11 12:15:59.104347 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.104276    2586 generic.go:356] "Generic (PLEG): container finished" podID="a2c95706-a2fb-4184-8dad-0631980ad76d" containerID="b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146" exitCode=0
Sep 11 12:15:59.104347 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.104283    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l" event={"ID":"a2c95706-a2fb-4184-8dad-0631980ad76d","Type":"ContainerDied","Data":"b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146"}
Sep 11 12:15:59.104534 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.104367    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l" event={"ID":"a2c95706-a2fb-4184-8dad-0631980ad76d","Type":"ContainerDied","Data":"c3264a490490ce3a0d7b1015afb7335bcde8d93e25b5c055f1870772906ec25d"}
Sep 11 12:15:59.104534 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.104292    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"
Sep 11 12:15:59.111785 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.111765    2586 scope.go:117] "RemoveContainer" containerID="a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f"
Sep 11 12:15:59.112054 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:59.112021    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f\": container with ID starting with a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f not found: ID does not exist" containerID="a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f"
Sep 11 12:15:59.112054 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.112047    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f"} err="failed to get container status \"a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f\": rpc error: code = NotFound desc = could not find container \"a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f\": container with ID starting with a7e038ad2c4b9bfc5d10c71dc1bcfb59b7c83a83bc29cc8920e8c05622890e0f not found: ID does not exist"
Sep 11 12:15:59.112192 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.112063    2586 scope.go:117] "RemoveContainer" containerID="b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146"
Sep 11 12:15:59.120062 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.120047    2586 scope.go:117] "RemoveContainer" containerID="b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146"
Sep 11 12:15:59.120323 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:15:59.120304    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146\": container with ID starting with b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146 not found: ID does not exist" containerID="b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146"
Sep 11 12:15:59.120379 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.120329    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146"} err="failed to get container status \"b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146\": rpc error: code = NotFound desc = could not find container \"b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146\": container with ID starting with b99abaff424b4e6fd18578fcd234014ec3eaf21aa8ecffa8126807aa3466f146 not found: ID does not exist"
Sep 11 12:15:59.131520 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.131475    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"]
Sep 11 12:15:59.138421 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.138397    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-50e434-59f64f77f7-z8gsk"]
Sep 11 12:15:59.149759 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.149740    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"]
Sep 11 12:15:59.154023 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:15:59.154001    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-50e434-79c7db7b98-5k96l"]
Sep 11 12:16:00.824337 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:00.824303    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35461298-3ebf-41bd-a4b1-ff7e2890e07c" path="/var/lib/kubelet/pods/35461298-3ebf-41bd-a4b1-ff7e2890e07c/volumes"
Sep 11 12:16:00.824712 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:00.824669    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c95706-a2fb-4184-8dad-0631980ad76d" path="/var/lib/kubelet/pods/a2c95706-a2fb-4184-8dad-0631980ad76d/volumes"
Sep 11 12:16:02.698363 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:02.698314    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:16:04.927206 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:04.927167    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:05.025728 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:05.025701    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:05.198706 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:05.198616    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:10.493448 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493412    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"]
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493782    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a9dee54-1ac6-40dc-b762-9e74cb635aaf" containerName="revoke-keys"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493796    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a9dee54-1ac6-40dc-b762-9e74cb635aaf" containerName="revoke-keys"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493820    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35461298-3ebf-41bd-a4b1-ff7e2890e07c" containerName="payload-processing"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493835    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="35461298-3ebf-41bd-a4b1-ff7e2890e07c" containerName="payload-processing"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493846    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c95706-a2fb-4184-8dad-0631980ad76d" containerName="payload-pre-processing"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493851    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c95706-a2fb-4184-8dad-0631980ad76d" containerName="payload-pre-processing"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493909    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a9dee54-1ac6-40dc-b762-9e74cb635aaf" containerName="revoke-keys"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493917    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c95706-a2fb-4184-8dad-0631980ad76d" containerName="payload-pre-processing"
Sep 11 12:16:10.493980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.493927    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="35461298-3ebf-41bd-a4b1-ff7e2890e07c" containerName="payload-processing"
Sep 11 12:16:10.504731 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.504703    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.507447 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.507423    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"]
Sep 11 12:16:10.507888 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.507869    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"kube-root-ca.crt\""
Sep 11 12:16:10.509668 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.509631    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"rate-test-model-4d3f83-kserve-self-signed-certs\""
Sep 11 12:16:10.509792 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.509744    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"default-dockercfg-6jstw\""
Sep 11 12:16:10.509792 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.509634    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4d3f83\"/\"openshift-service-ca.crt\""
Sep 11 12:16:10.528441 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.528412    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:10.647226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.647202    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-model-cache\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.647316 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.647252    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tmp-dir\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.647316 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.647289    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-home\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.647396 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.647326    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-dshm\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.647396 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.647381    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t29bb\" (UniqueName: \"kubernetes.io/projected/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-kube-api-access-t29bb\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.647462 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.647422    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tls-certs\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.747942 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.747875    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tmp-dir\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.747942 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.747908    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-home\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.747942 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.747924    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-dshm\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.748158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.747969    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t29bb\" (UniqueName: \"kubernetes.io/projected/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-kube-api-access-t29bb\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.748158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.748016    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tls-certs\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.748158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.748059    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-model-cache\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.748315 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.748297    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tmp-dir\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.748415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.748390    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-home\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.748472 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.748434    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-model-cache\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.750470 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.750449    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-dshm\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.750774 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.750755    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tls-certs\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.757027 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.757006    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t29bb\" (UniqueName: \"kubernetes.io/projected/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-kube-api-access-t29bb\") pod \"rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") " pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:10.818070 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:10.818044    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:11.094137 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:11.094112    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"]
Sep 11 12:16:11.095462 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:16:11.095437    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4ff3601_5b4f_4b2d_b635_581d01fa2b1d.slice/crio-2a323cb36b825afa78c95d5137d333c7844d399d896f587cb8e2c404a8d0f4c9 WatchSource:0}: Error finding container 2a323cb36b825afa78c95d5137d333c7844d399d896f587cb8e2c404a8d0f4c9: Status 404 returned error can't find the container with id 2a323cb36b825afa78c95d5137d333c7844d399d896f587cb8e2c404a8d0f4c9
Sep 11 12:16:11.170744 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:11.170710    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44" event={"ID":"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d","Type":"ContainerStarted","Data":"499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933"}
Sep 11 12:16:11.170870 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:11.170762    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44" event={"ID":"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d","Type":"ContainerStarted","Data":"2a323cb36b825afa78c95d5137d333c7844d399d896f587cb8e2c404a8d0f4c9"}
Sep 11 12:16:11.188927 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:11.188885    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44" podStartSLOduration=1.188869216 podStartE2EDuration="1.188869216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:16:11.188003874 +0000 UTC m=+1464.898675212" watchObservedRunningTime="2026-09-11 12:16:11.188869216 +0000 UTC m=+1464.899540560"
Sep 11 12:16:15.005915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:15.005870    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:16:20.823696 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:20.823661    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:20.824066 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:20.823709    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:20.832055 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:20.832029    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:21.231802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:21.231729    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:16:33.532698 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:33.532662    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:16:44.502070 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:44.502032    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:16:46.871828 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:46.871802    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:16:46.877184 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:46.877161    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:16:51.486134 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:51.486104    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"]
Sep 11 12:16:52.595694 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:52.595658    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"]
Sep 11 12:16:52.774108 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:52.774076    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:53.458099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:53.458066    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:53.558218 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:53.558193    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:56.897591 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:56.897558    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:56.997542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:56.997512    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:57.408124 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:57.408089    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:16:58.064568 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.064537    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:58.164814 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.164783    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:58.364223 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.364147    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:58.652787 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.652704    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:58.731690 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.731654    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:16:58.732693 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.732673    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"]
Sep 11 12:16:58.737796 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.737776    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:58.740812 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.740790    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-9c3012\""
Sep 11 12:16:58.747769 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.747731    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"]
Sep 11 12:16:58.778670 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.778643    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"]
Sep 11 12:16:58.782132 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.782117    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:58.785148 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.785131    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-9c3012-dockercfg-nnqmx\""
Sep 11 12:16:58.792421 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.792396    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"]
Sep 11 12:16:58.838560 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.838510    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99da832-a425-4b34-8559-c09ea93d0367-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:58.838748 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.838684    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g9wk\" (UniqueName: \"kubernetes.io/projected/56b74704-8198-4785-a39e-430206e1e0a5-kube-api-access-5g9wk\") pod \"payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:58.838825 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.838808    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b74704-8198-4785-a39e-430206e1e0a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:58.838961 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.838941    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkzj7\" (UniqueName: \"kubernetes.io/projected/d99da832-a425-4b34-8559-c09ea93d0367-kube-api-access-mkzj7\") pod \"payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:58.940287 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.940217    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkzj7\" (UniqueName: \"kubernetes.io/projected/d99da832-a425-4b34-8559-c09ea93d0367-kube-api-access-mkzj7\") pod \"payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:58.940287 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.940257    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99da832-a425-4b34-8559-c09ea93d0367-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:58.940287 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.940282    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g9wk\" (UniqueName: \"kubernetes.io/projected/56b74704-8198-4785-a39e-430206e1e0a5-kube-api-access-5g9wk\") pod \"payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:58.940584 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.940310    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b74704-8198-4785-a39e-430206e1e0a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:58.940902 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.940883    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99da832-a425-4b34-8559-c09ea93d0367-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:58.940949 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.940924    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b74704-8198-4785-a39e-430206e1e0a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:58.952772 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.952752    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g9wk\" (UniqueName: \"kubernetes.io/projected/56b74704-8198-4785-a39e-430206e1e0a5-kube-api-access-5g9wk\") pod \"payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:58.952894 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:58.952880    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkzj7\" (UniqueName: \"kubernetes.io/projected/d99da832-a425-4b34-8559-c09ea93d0367-kube-api-access-mkzj7\") pod \"payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:59.049648 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.049626    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:59.093632 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.093605    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:16:59.321633 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.321606    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"]
Sep 11 12:16:59.323335 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:16:59.323313    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b74704_8198_4785_a39e_430206e1e0a5.slice/crio-2bd88b152b56399074086390e3b9c6b8cfe9560272471c8b12005bf0e6de8f29 WatchSource:0}: Error finding container 2bd88b152b56399074086390e3b9c6b8cfe9560272471c8b12005bf0e6de8f29: Status 404 returned error can't find the container with id 2bd88b152b56399074086390e3b9c6b8cfe9560272471c8b12005bf0e6de8f29
Sep 11 12:16:59.381741 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.381718    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"]
Sep 11 12:16:59.383406 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:16:59.383383    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99da832_a425_4b34_8559_c09ea93d0367.slice/crio-4a4a4bd4b9250aad2857aca373f2551ba7e78e41492e7ab2af7838a7c4991e99 WatchSource:0}: Error finding container 4a4a4bd4b9250aad2857aca373f2551ba7e78e41492e7ab2af7838a7c4991e99: Status 404 returned error can't find the container with id 4a4a4bd4b9250aad2857aca373f2551ba7e78e41492e7ab2af7838a7c4991e99
Sep 11 12:16:59.415921 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.415864    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r" event={"ID":"d99da832-a425-4b34-8559-c09ea93d0367","Type":"ContainerStarted","Data":"4a4a4bd4b9250aad2857aca373f2551ba7e78e41492e7ab2af7838a7c4991e99"}
Sep 11 12:16:59.418099 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.418068    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh" event={"ID":"56b74704-8198-4785-a39e-430206e1e0a5","Type":"ContainerStarted","Data":"8a2bb431492279565ec6d35957bdba891648c9a929fd253c7305742f2e1e3940"}
Sep 11 12:16:59.418211 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.418120    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh" event={"ID":"56b74704-8198-4785-a39e-430206e1e0a5","Type":"ContainerStarted","Data":"2bd88b152b56399074086390e3b9c6b8cfe9560272471c8b12005bf0e6de8f29"}
Sep 11 12:16:59.418211 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.418190    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:16:59.435848 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:16:59.435654    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh" podStartSLOduration=1.435632729 podStartE2EDuration="1.435632729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:16:59.434812416 +0000 UTC m=+1513.145483765" watchObservedRunningTime="2026-09-11 12:16:59.435632729 +0000 UTC m=+1513.146304070"
Sep 11 12:17:00.425513 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:00.425451    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r" event={"ID":"d99da832-a425-4b34-8559-c09ea93d0367","Type":"ContainerStarted","Data":"33d6d67018432ddd4f5c79c4400011dde5e9e2db39ada55b31ad58f503374e84"}
Sep 11 12:17:00.427935 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:00.425645    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:17:00.443679 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:00.443625    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r" podStartSLOduration=2.443607104 podStartE2EDuration="2.443607104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:17:00.44222128 +0000 UTC m=+1514.152892623" watchObservedRunningTime="2026-09-11 12:17:00.443607104 +0000 UTC m=+1514.154278447"
Sep 11 12:17:06.507282 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.507243    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx" podUID="b29ba867-3bd2-49a3-b740-28bda9659786" containerName="main" containerID="cri-o://d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa" gracePeriod=45
Sep 11 12:17:06.767598 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.767541    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:17:06.913923 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.913888    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-tmp-dir\") pod \"b29ba867-3bd2-49a3-b740-28bda9659786\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") "
Sep 11 12:17:06.914113 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.913954    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5sgg\" (UniqueName: \"kubernetes.io/projected/b29ba867-3bd2-49a3-b740-28bda9659786-kube-api-access-t5sgg\") pod \"b29ba867-3bd2-49a3-b740-28bda9659786\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") "
Sep 11 12:17:06.914113 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914043    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-home\") pod \"b29ba867-3bd2-49a3-b740-28bda9659786\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") "
Sep 11 12:17:06.914113 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914093    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-model-cache\") pod \"b29ba867-3bd2-49a3-b740-28bda9659786\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") "
Sep 11 12:17:06.914283 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914116    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29ba867-3bd2-49a3-b740-28bda9659786-tls-certs\") pod \"b29ba867-3bd2-49a3-b740-28bda9659786\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") "
Sep 11 12:17:06.914283 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914144    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-dshm\") pod \"b29ba867-3bd2-49a3-b740-28bda9659786\" (UID: \"b29ba867-3bd2-49a3-b740-28bda9659786\") "
Sep 11 12:17:06.914283 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914144    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b29ba867-3bd2-49a3-b740-28bda9659786" (UID: "b29ba867-3bd2-49a3-b740-28bda9659786"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:06.914420 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914297    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-home" (OuterVolumeSpecName: "home") pod "b29ba867-3bd2-49a3-b740-28bda9659786" (UID: "b29ba867-3bd2-49a3-b740-28bda9659786"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:06.914420 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914342    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-model-cache" (OuterVolumeSpecName: "model-cache") pod "b29ba867-3bd2-49a3-b740-28bda9659786" (UID: "b29ba867-3bd2-49a3-b740-28bda9659786"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:06.914528 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914472    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:06.914528 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914517    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:06.914609 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.914545    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:06.916281 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.916251    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-dshm" (OuterVolumeSpecName: "dshm") pod "b29ba867-3bd2-49a3-b740-28bda9659786" (UID: "b29ba867-3bd2-49a3-b740-28bda9659786"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:06.916281 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.916272    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b29ba867-3bd2-49a3-b740-28bda9659786-kube-api-access-t5sgg" (OuterVolumeSpecName: "kube-api-access-t5sgg") pod "b29ba867-3bd2-49a3-b740-28bda9659786" (UID: "b29ba867-3bd2-49a3-b740-28bda9659786"). InnerVolumeSpecName "kube-api-access-t5sgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:17:06.916468 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:06.916322    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b29ba867-3bd2-49a3-b740-28bda9659786-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b29ba867-3bd2-49a3-b740-28bda9659786" (UID: "b29ba867-3bd2-49a3-b740-28bda9659786"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:17:07.015570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.015533    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5sgg\" (UniqueName: \"kubernetes.io/projected/b29ba867-3bd2-49a3-b740-28bda9659786-kube-api-access-t5sgg\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:07.015570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.015568    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29ba867-3bd2-49a3-b740-28bda9659786-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:07.015570 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.015579    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29ba867-3bd2-49a3-b740-28bda9659786-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:07.461637 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.461600    2586 generic.go:356] "Generic (PLEG): container finished" podID="b29ba867-3bd2-49a3-b740-28bda9659786" containerID="d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa" exitCode=0
Sep 11 12:17:07.461811 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.461644    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"
Sep 11 12:17:07.461811 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.461685    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx" event={"ID":"b29ba867-3bd2-49a3-b740-28bda9659786","Type":"ContainerDied","Data":"d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa"}
Sep 11 12:17:07.461811 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.461727    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx" event={"ID":"b29ba867-3bd2-49a3-b740-28bda9659786","Type":"ContainerDied","Data":"96c0111bbf1530aa7ff276717300faa888cf522a295e9126b55b071a1b33a490"}
Sep 11 12:17:07.461811 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.461744    2586 scope.go:117] "RemoveContainer" containerID="d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa"
Sep 11 12:17:07.470510 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.470471    2586 scope.go:117] "RemoveContainer" containerID="d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa"
Sep 11 12:17:07.470775 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:17:07.470754    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa\": container with ID starting with d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa not found: ID does not exist" containerID="d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa"
Sep 11 12:17:07.470829 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.470783    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa"} err="failed to get container status \"d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa\": rpc error: code = NotFound desc = could not find container \"d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa\": container with ID starting with d45b9667c890130bff5e626a524f2ae8e183ecfc3cf39c9f8e57569af04bedaa not found: ID does not exist"
Sep 11 12:17:07.482842 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.482820    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"]
Sep 11 12:17:07.486637 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.486617    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-88968d/rate-test-model-88968d-kserve-78849459c5-9zhbx"]
Sep 11 12:17:07.615496 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.615459    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44" podUID="c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" containerName="main" containerID="cri-o://499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933" gracePeriod=45
Sep 11 12:17:07.869060 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.869033    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:17:07.925134 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925104    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-home\") pod \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") "
Sep 11 12:17:07.925307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925159    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-dshm\") pod \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") "
Sep 11 12:17:07.925307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925193    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-model-cache\") pod \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") "
Sep 11 12:17:07.925307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925226    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tmp-dir\") pod \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") "
Sep 11 12:17:07.925307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925250    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tls-certs\") pod \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") "
Sep 11 12:17:07.925307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925287    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t29bb\" (UniqueName: \"kubernetes.io/projected/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-kube-api-access-t29bb\") pod \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\" (UID: \"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d\") "
Sep 11 12:17:07.925613 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925432    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-home" (OuterVolumeSpecName: "home") pod "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" (UID: "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:07.925613 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925508    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" (UID: "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:07.925613 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925569    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" (UID: "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:07.925730 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925673    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:07.925730 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925688    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:07.925730 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.925701    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:07.927526 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.927499    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" (UID: "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:17:07.927526 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.927511    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-kube-api-access-t29bb" (OuterVolumeSpecName: "kube-api-access-t29bb") pod "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" (UID: "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d"). InnerVolumeSpecName "kube-api-access-t29bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:17:07.927673 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:07.927534    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-dshm" (OuterVolumeSpecName: "dshm") pod "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" (UID: "c4ff3601-5b4f-4b2d-b635-581d01fa2b1d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:17:08.026746 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.026670    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:08.026746 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.026695    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t29bb\" (UniqueName: \"kubernetes.io/projected/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-kube-api-access-t29bb\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:08.026746 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.026706    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:17:08.468828 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.468791    2586 generic.go:356] "Generic (PLEG): container finished" podID="c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" containerID="499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933" exitCode=0
Sep 11 12:17:08.469017 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.468840    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"
Sep 11 12:17:08.469017 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.468848    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44" event={"ID":"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d","Type":"ContainerDied","Data":"499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933"}
Sep 11 12:17:08.469017 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.468885    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44" event={"ID":"c4ff3601-5b4f-4b2d-b635-581d01fa2b1d","Type":"ContainerDied","Data":"2a323cb36b825afa78c95d5137d333c7844d399d896f587cb8e2c404a8d0f4c9"}
Sep 11 12:17:08.469017 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.468908    2586 scope.go:117] "RemoveContainer" containerID="499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933"
Sep 11 12:17:08.477877 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.477860    2586 scope.go:117] "RemoveContainer" containerID="499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933"
Sep 11 12:17:08.478139 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:17:08.478121    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933\": container with ID starting with 499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933 not found: ID does not exist" containerID="499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933"
Sep 11 12:17:08.478188 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.478147    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933"} err="failed to get container status \"499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933\": rpc error: code = NotFound desc = could not find container \"499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933\": container with ID starting with 499a0bd35d59cfc51ba16928231ec97512596223f1672b9640ff0b94bdce3933 not found: ID does not exist"
Sep 11 12:17:08.489460 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.489434    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"]
Sep 11 12:17:08.494748 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.494726    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4d3f83/rate-test-model-4d3f83-kserve-5cfb9cffc9-vch44"]
Sep 11 12:17:08.823229 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.823198    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b29ba867-3bd2-49a3-b740-28bda9659786" path="/var/lib/kubelet/pods/b29ba867-3bd2-49a3-b740-28bda9659786/volumes"
Sep 11 12:17:08.823776 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:08.823756    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" path="/var/lib/kubelet/pods/c4ff3601-5b4f-4b2d-b635-581d01fa2b1d/volumes"
Sep 11 12:17:09.510717 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:09.510683    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:17:10.427594 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:10.427561    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:17:11.432693 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:11.432665    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:17:21.737013 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:21.736973    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:22.405120 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:22.405085    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:22.689764 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:22.689661    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:22.833920 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:22.833889    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:26.113472 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:26.113434    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:26.212567 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:26.212532    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:27.262211 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:27.262183    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:27.367424 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:27.367385    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:27.500153 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:27.500122    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:27.853148 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:27.853117    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:27.950175 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:27.950144    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:28.035563 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.035531    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"]
Sep 11 12:17:28.035904 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.035892    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b29ba867-3bd2-49a3-b740-28bda9659786" containerName="main"
Sep 11 12:17:28.035957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.035905    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b29ba867-3bd2-49a3-b740-28bda9659786" containerName="main"
Sep 11 12:17:28.035957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.035924    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" containerName="main"
Sep 11 12:17:28.035957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.035929    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" containerName="main"
Sep 11 12:17:28.036055 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.035983    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ff3601-5b4f-4b2d-b635-581d01fa2b1d" containerName="main"
Sep 11 12:17:28.036055 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.035993    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b29ba867-3bd2-49a3-b740-28bda9659786" containerName="main"
Sep 11 12:17:28.038806 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.038790    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.042017 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.041988    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6d324e-dockercfg-ngzkc\""
Sep 11 12:17:28.042906 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.042878    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6d324e\""
Sep 11 12:17:28.054653 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.054631    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"]
Sep 11 12:17:28.104577 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.104513    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0655b-4811-4541-9885-7aa8c28e3b33-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.104682 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.104573    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9227\" (UniqueName: \"kubernetes.io/projected/e6a0655b-4811-4541-9885-7aa8c28e3b33-kube-api-access-t9227\") pod \"payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.156280 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.156249    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"]
Sep 11 12:17:28.159749 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.159733    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.169026 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.168996    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"]
Sep 11 12:17:28.205837 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.205811    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0655b-4811-4541-9885-7aa8c28e3b33-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.205981 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.205847    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6e3590-c248-4437-814d-5ec4853ac591-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.205981 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.205899    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9227\" (UniqueName: \"kubernetes.io/projected/e6a0655b-4811-4541-9885-7aa8c28e3b33-kube-api-access-t9227\") pod \"payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.205981 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.205932    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82bmv\" (UniqueName: \"kubernetes.io/projected/4f6e3590-c248-4437-814d-5ec4853ac591-kube-api-access-82bmv\") pod \"payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.206455 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.206436    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0655b-4811-4541-9885-7aa8c28e3b33-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.214953 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.214929    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9227\" (UniqueName: \"kubernetes.io/projected/e6a0655b-4811-4541-9885-7aa8c28e3b33-kube-api-access-t9227\") pod \"payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.307244 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.307208    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6e3590-c248-4437-814d-5ec4853ac591-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.307741 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.307274    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82bmv\" (UniqueName: \"kubernetes.io/projected/4f6e3590-c248-4437-814d-5ec4853ac591-kube-api-access-82bmv\") pod \"payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.308067 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.308042    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6e3590-c248-4437-814d-5ec4853ac591-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.316052 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.316025    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82bmv\" (UniqueName: \"kubernetes.io/projected/4f6e3590-c248-4437-814d-5ec4853ac591-kube-api-access-82bmv\") pod \"payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.348749 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.348725    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:28.471042 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.471012    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:28.622852 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.622824    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"]
Sep 11 12:17:28.624881 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:17:28.624855    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a0655b_4811_4541_9885_7aa8c28e3b33.slice/crio-0db53d398e642526a4359a7fe3f3ce5e5ef21375d26234d865a0954e604f569b WatchSource:0}: Error finding container 0db53d398e642526a4359a7fe3f3ce5e5ef21375d26234d865a0954e604f569b: Status 404 returned error can't find the container with id 0db53d398e642526a4359a7fe3f3ce5e5ef21375d26234d865a0954e604f569b
Sep 11 12:17:28.746855 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:28.746817    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"]
Sep 11 12:17:28.748403 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:17:28.748373    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6e3590_c248_4437_814d_5ec4853ac591.slice/crio-e34ae3990b395d8dfcfa4ebb66ef872088c5fc33d43ea4ae4912ee85ca0500e8 WatchSource:0}: Error finding container e34ae3990b395d8dfcfa4ebb66ef872088c5fc33d43ea4ae4912ee85ca0500e8: Status 404 returned error can't find the container with id e34ae3990b395d8dfcfa4ebb66ef872088c5fc33d43ea4ae4912ee85ca0500e8
Sep 11 12:17:29.578369 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:29.578331    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t" event={"ID":"4f6e3590-c248-4437-814d-5ec4853ac591","Type":"ContainerStarted","Data":"6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88"}
Sep 11 12:17:29.578840 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:29.578394    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:29.578840 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:29.578418    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t" event={"ID":"4f6e3590-c248-4437-814d-5ec4853ac591","Type":"ContainerStarted","Data":"e34ae3990b395d8dfcfa4ebb66ef872088c5fc33d43ea4ae4912ee85ca0500e8"}
Sep 11 12:17:29.580016 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:29.579986    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px" event={"ID":"e6a0655b-4811-4541-9885-7aa8c28e3b33","Type":"ContainerStarted","Data":"70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89"}
Sep 11 12:17:29.580141 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:29.580031    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px" event={"ID":"e6a0655b-4811-4541-9885-7aa8c28e3b33","Type":"ContainerStarted","Data":"0db53d398e642526a4359a7fe3f3ce5e5ef21375d26234d865a0954e604f569b"}
Sep 11 12:17:29.580141 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:29.580052    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:29.596705 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:29.596665    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t" podStartSLOduration=1.5966504929999998 podStartE2EDuration="1.596650493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:17:29.595608697 +0000 UTC m=+1543.306280035" watchObservedRunningTime="2026-09-11 12:17:29.596650493 +0000 UTC m=+1543.307321832"
Sep 11 12:17:33.599703 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:33.599644    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px" podStartSLOduration=5.599622698 podStartE2EDuration="5.599622698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:17:29.612251236 +0000 UTC m=+1543.322922574" watchObservedRunningTime="2026-09-11 12:17:33.599622698 +0000 UTC m=+1547.310294037"
Sep 11 12:17:33.600403 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:33.600381    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:17:40.586857 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:40.586814    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:17:40.587269 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:40.586901    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:17:50.964196 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:50.964156    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:17:56.009191 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:17:56.009157    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t_4f6e3590-c248-4437-814d-5ec4853ac591/payload-processing/0.log"
Sep 11 12:18:00.997984 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:00.997950    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"]
Sep 11 12:18:01.003130 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.003106    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.007291 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.007264    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-9c3012\"/\"kube-root-ca.crt\""
Sep 11 12:18:01.007291 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.007280    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-9c3012\"/\"openshift-service-ca.crt\""
Sep 11 12:18:01.007517 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.007365    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-9c3012\"/\"default-dockercfg-677ng\""
Sep 11 12:18:01.007517 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.007387    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-9c3012\"/\"ipp-route-9c3012-kserve-self-signed-certs\""
Sep 11 12:18:01.012914 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.012888    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"]
Sep 11 12:18:01.032683 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.032644    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:01.099425 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.099401    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szdfx\" (UniqueName: \"kubernetes.io/projected/45bea232-ccc0-4377-9abb-91053ca43096-kube-api-access-szdfx\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.099554 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.099447    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-tmp-dir\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.099554 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.099479    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bea232-ccc0-4377-9abb-91053ca43096-tls-certs\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.099554 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.099519    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-model-cache\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.099554 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.099547    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-home\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.099717 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.099625    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-dshm\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.200957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.200933    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-dshm\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201054 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.200974    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szdfx\" (UniqueName: \"kubernetes.io/projected/45bea232-ccc0-4377-9abb-91053ca43096-kube-api-access-szdfx\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201116 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.201097    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-tmp-dir\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201173 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.201158    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bea232-ccc0-4377-9abb-91053ca43096-tls-certs\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201211 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.201200    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-model-cache\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201268 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.201249    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-home\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201537 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.201516    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-tmp-dir\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.201519    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-model-cache\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.201615 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.201577    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-home\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.203218 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.203196    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-dshm\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.203639 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.203625    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bea232-ccc0-4377-9abb-91053ca43096-tls-certs\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.209584 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.209564    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szdfx\" (UniqueName: \"kubernetes.io/projected/45bea232-ccc0-4377-9abb-91053ca43096-kube-api-access-szdfx\") pod \"ipp-route-9c3012-kserve-6c596fc87-f98xx\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") " pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.314357 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.314330    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:01.585535 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.585478    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"]
Sep 11 12:18:01.587935 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:18:01.587897    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45bea232_ccc0_4377_9abb_91053ca43096.slice/crio-fec963ab501ce0c94599d0455c7f5c49d1ceb6d9d39f16d2446e0a4d86f7fd47 WatchSource:0}: Error finding container fec963ab501ce0c94599d0455c7f5c49d1ceb6d9d39f16d2446e0a4d86f7fd47: Status 404 returned error can't find the container with id fec963ab501ce0c94599d0455c7f5c49d1ceb6d9d39f16d2446e0a4d86f7fd47
Sep 11 12:18:01.747014 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.746973    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx" event={"ID":"45bea232-ccc0-4377-9abb-91053ca43096","Type":"ContainerStarted","Data":"929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c"}
Sep 11 12:18:01.747163 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:01.747023    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx" event={"ID":"45bea232-ccc0-4377-9abb-91053ca43096","Type":"ContainerStarted","Data":"fec963ab501ce0c94599d0455c7f5c49d1ceb6d9d39f16d2446e0a4d86f7fd47"}
Sep 11 12:18:05.197571 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:05.197510    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx" podStartSLOduration=5.197466977 podStartE2EDuration="5.197466977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:18:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:18:01.762953279 +0000 UTC m=+1575.473624618" watchObservedRunningTime="2026-09-11 12:18:05.197466977 +0000 UTC m=+1578.908138344"
Sep 11 12:18:05.199314 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:05.199287    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:18:11.315338 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:11.315308    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:11.315769 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:11.315355    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:11.328043 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:11.328017    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:11.812085 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:11.812054    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:18:20.203818 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:20.203782    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:18:26.400391 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:26.400347    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r_d99da832-a425-4b34-8559-c09ea93d0367/payload-processing/0.log"
Sep 11 12:18:26.637034 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:26.637008    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t_4f6e3590-c248-4437-814d-5ec4853ac591/payload-processing/0.log"
Sep 11 12:18:26.904047 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:26.904015    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:27.199775 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:27.199692    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:27.524293 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:27.524218    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:30.409125 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:30.409087    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:30.507452 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:30.507423    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:31.683389 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:31.683360    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:31.753013 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:31.752978    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:31.904258 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:31.904228    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:32.248608 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.248571    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:32.347301 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.347272    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:32.361915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.361886    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"]
Sep 11 12:18:32.365580 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.365564    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.368415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.368395    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-49995f-dockercfg-w2skm\""
Sep 11 12:18:32.368542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.368397    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-49995f\""
Sep 11 12:18:32.375394 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.375374    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"]
Sep 11 12:18:32.403727 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.403705    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"]
Sep 11 12:18:32.407039 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.407025    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.427152 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.427130    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"]
Sep 11 12:18:32.454948 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.454922    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205afe17-9751-462a-8269-7d57e89136ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.455043 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.454969    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q5kj\" (UniqueName: \"kubernetes.io/projected/205afe17-9751-462a-8269-7d57e89136ab-kube-api-access-7q5kj\") pod \"payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.555960 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.555937    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0569cf5f-de74-4245-95f5-9168393902b6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.556051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.555974    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205afe17-9751-462a-8269-7d57e89136ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.556051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.556012    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q5kj\" (UniqueName: \"kubernetes.io/projected/205afe17-9751-462a-8269-7d57e89136ab-kube-api-access-7q5kj\") pod \"payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.556147 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.556060    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krnxw\" (UniqueName: \"kubernetes.io/projected/0569cf5f-de74-4245-95f5-9168393902b6-kube-api-access-krnxw\") pod \"payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.556670 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.556653    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205afe17-9751-462a-8269-7d57e89136ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.577629 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.577606    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q5kj\" (UniqueName: \"kubernetes.io/projected/205afe17-9751-462a-8269-7d57e89136ab-kube-api-access-7q5kj\") pod \"payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.657070 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.657051    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krnxw\" (UniqueName: \"kubernetes.io/projected/0569cf5f-de74-4245-95f5-9168393902b6-kube-api-access-krnxw\") pod \"payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.657179 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.657088    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0569cf5f-de74-4245-95f5-9168393902b6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.657674 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.657658    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0569cf5f-de74-4245-95f5-9168393902b6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.665511 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.665466    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krnxw\" (UniqueName: \"kubernetes.io/projected/0569cf5f-de74-4245-95f5-9168393902b6-kube-api-access-krnxw\") pod \"payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.677139 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.677119    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:32.716833 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.716804    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:32.954092 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:32.954066    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"]
Sep 11 12:18:32.955966 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:18:32.955915    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205afe17_9751_462a_8269_7d57e89136ab.slice/crio-36b644028ddbc53f364396eb77c0fa6da470dd3f66e23e5ae84a9e0037fec68f WatchSource:0}: Error finding container 36b644028ddbc53f364396eb77c0fa6da470dd3f66e23e5ae84a9e0037fec68f: Status 404 returned error can't find the container with id 36b644028ddbc53f364396eb77c0fa6da470dd3f66e23e5ae84a9e0037fec68f
Sep 11 12:18:33.006195 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.006172    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"]
Sep 11 12:18:33.007821 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:18:33.007783    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0569cf5f_de74_4245_95f5_9168393902b6.slice/crio-56de99d19c3cfee23af787b2dc566e318dbc4f0eb3ef579c9530fcb0e1bfcf82 WatchSource:0}: Error finding container 56de99d19c3cfee23af787b2dc566e318dbc4f0eb3ef579c9530fcb0e1bfcf82: Status 404 returned error can't find the container with id 56de99d19c3cfee23af787b2dc566e318dbc4f0eb3ef579c9530fcb0e1bfcf82
Sep 11 12:18:33.914000 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.913962    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69" event={"ID":"0569cf5f-de74-4245-95f5-9168393902b6","Type":"ContainerStarted","Data":"0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330"}
Sep 11 12:18:33.914446 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.914016    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69" event={"ID":"0569cf5f-de74-4245-95f5-9168393902b6","Type":"ContainerStarted","Data":"56de99d19c3cfee23af787b2dc566e318dbc4f0eb3ef579c9530fcb0e1bfcf82"}
Sep 11 12:18:33.914446 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.914050    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:33.915592 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.915566    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj" event={"ID":"205afe17-9751-462a-8269-7d57e89136ab","Type":"ContainerStarted","Data":"9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6"}
Sep 11 12:18:33.915711 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.915615    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj" event={"ID":"205afe17-9751-462a-8269-7d57e89136ab","Type":"ContainerStarted","Data":"36b644028ddbc53f364396eb77c0fa6da470dd3f66e23e5ae84a9e0037fec68f"}
Sep 11 12:18:33.915711 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.915644    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:33.937992 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.937935    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69" podStartSLOduration=1.93791447 podStartE2EDuration="1.93791447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:18:33.935882018 +0000 UTC m=+1607.646553357" watchObservedRunningTime="2026-09-11 12:18:33.93791447 +0000 UTC m=+1607.648585809"
Sep 11 12:18:33.955741 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:33.955700    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj" podStartSLOduration=1.955686954 podStartE2EDuration="1.955686954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:18:33.95407568 +0000 UTC m=+1607.664747020" watchObservedRunningTime="2026-09-11 12:18:33.955686954 +0000 UTC m=+1607.666358291"
Sep 11 12:18:36.391763 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:36.391726    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:39.880143 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:39.880047    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:18:40.503852 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:40.503814    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:18:44.923396 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:44.923365    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:18:44.923863 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:44.923433    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:18:55.364515 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:18:55.364461    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:02.103252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.103222    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:02.138731 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.138697    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"]
Sep 11 12:19:02.139027 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.138947    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69" podUID="0569cf5f-de74-4245-95f5-9168393902b6" containerName="payload-processing" containerID="cri-o://0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330" gracePeriod=30
Sep 11 12:19:02.148464 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.148436    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"]
Sep 11 12:19:02.148802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.148759    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj" podUID="205afe17-9751-462a-8269-7d57e89136ab" containerName="payload-pre-processing" containerID="cri-o://9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6" gracePeriod=30
Sep 11 12:19:02.185472 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.185416    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:02.246376 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.246346    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:02.310849 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.310759    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:02.382598 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.382565    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:02.401265 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.401240    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:19:02.412949 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.412927    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0569cf5f-de74-4245-95f5-9168393902b6-plugins-config-volume\") pod \"0569cf5f-de74-4245-95f5-9168393902b6\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") "
Sep 11 12:19:02.413057 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.412984    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krnxw\" (UniqueName: \"kubernetes.io/projected/0569cf5f-de74-4245-95f5-9168393902b6-kube-api-access-krnxw\") pod \"0569cf5f-de74-4245-95f5-9168393902b6\" (UID: \"0569cf5f-de74-4245-95f5-9168393902b6\") "
Sep 11 12:19:02.413310 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.413284    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0569cf5f-de74-4245-95f5-9168393902b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0569cf5f-de74-4245-95f5-9168393902b6" (UID: "0569cf5f-de74-4245-95f5-9168393902b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:02.415149 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.415128    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0569cf5f-de74-4245-95f5-9168393902b6-kube-api-access-krnxw" (OuterVolumeSpecName: "kube-api-access-krnxw") pod "0569cf5f-de74-4245-95f5-9168393902b6" (UID: "0569cf5f-de74-4245-95f5-9168393902b6"). InnerVolumeSpecName "kube-api-access-krnxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:02.432685 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.432666    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:19:02.513947 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.513925    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205afe17-9751-462a-8269-7d57e89136ab-plugins-config-volume\") pod \"205afe17-9751-462a-8269-7d57e89136ab\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") "
Sep 11 12:19:02.514063 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.514014    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q5kj\" (UniqueName: \"kubernetes.io/projected/205afe17-9751-462a-8269-7d57e89136ab-kube-api-access-7q5kj\") pod \"205afe17-9751-462a-8269-7d57e89136ab\" (UID: \"205afe17-9751-462a-8269-7d57e89136ab\") "
Sep 11 12:19:02.514208 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.514194    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0569cf5f-de74-4245-95f5-9168393902b6-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.514285 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.514209    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krnxw\" (UniqueName: \"kubernetes.io/projected/0569cf5f-de74-4245-95f5-9168393902b6-kube-api-access-krnxw\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.514285 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.514256    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205afe17-9751-462a-8269-7d57e89136ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "205afe17-9751-462a-8269-7d57e89136ab" (UID: "205afe17-9751-462a-8269-7d57e89136ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:02.516167 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.516146    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205afe17-9751-462a-8269-7d57e89136ab-kube-api-access-7q5kj" (OuterVolumeSpecName: "kube-api-access-7q5kj") pod "205afe17-9751-462a-8269-7d57e89136ab" (UID: "205afe17-9751-462a-8269-7d57e89136ab"). InnerVolumeSpecName "kube-api-access-7q5kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:02.614848 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.614794    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205afe17-9751-462a-8269-7d57e89136ab-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.614848 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.614817    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q5kj\" (UniqueName: \"kubernetes.io/projected/205afe17-9751-462a-8269-7d57e89136ab-kube-api-access-7q5kj\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.666640 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:02.666610    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:03.066619 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.066585    2586 generic.go:356] "Generic (PLEG): container finished" podID="0569cf5f-de74-4245-95f5-9168393902b6" containerID="0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330" exitCode=0
Sep 11 12:19:03.066781 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.066633    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"
Sep 11 12:19:03.066781 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.066676    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69" event={"ID":"0569cf5f-de74-4245-95f5-9168393902b6","Type":"ContainerDied","Data":"0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330"}
Sep 11 12:19:03.066781 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.066765    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69" event={"ID":"0569cf5f-de74-4245-95f5-9168393902b6","Type":"ContainerDied","Data":"56de99d19c3cfee23af787b2dc566e318dbc4f0eb3ef579c9530fcb0e1bfcf82"}
Sep 11 12:19:03.066967 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.066796    2586 scope.go:117] "RemoveContainer" containerID="0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330"
Sep 11 12:19:03.068052 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.068032    2586 generic.go:356] "Generic (PLEG): container finished" podID="205afe17-9751-462a-8269-7d57e89136ab" containerID="9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6" exitCode=0
Sep 11 12:19:03.068148 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.068064    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj" event={"ID":"205afe17-9751-462a-8269-7d57e89136ab","Type":"ContainerDied","Data":"9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6"}
Sep 11 12:19:03.068148 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.068101    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"
Sep 11 12:19:03.068148 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.068119    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj" event={"ID":"205afe17-9751-462a-8269-7d57e89136ab","Type":"ContainerDied","Data":"36b644028ddbc53f364396eb77c0fa6da470dd3f66e23e5ae84a9e0037fec68f"}
Sep 11 12:19:03.075704 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.075688    2586 scope.go:117] "RemoveContainer" containerID="0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330"
Sep 11 12:19:03.075963 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:19:03.075943    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330\": container with ID starting with 0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330 not found: ID does not exist" containerID="0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330"
Sep 11 12:19:03.076010 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.075971    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330"} err="failed to get container status \"0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330\": rpc error: code = NotFound desc = could not find container \"0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330\": container with ID starting with 0fe117b815f6af4e2328a20a824aa910bccfb6da0d48131db7fdbdc3c0661330 not found: ID does not exist"
Sep 11 12:19:03.076010 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.075987    2586 scope.go:117] "RemoveContainer" containerID="9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6"
Sep 11 12:19:03.083850 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.083833    2586 scope.go:117] "RemoveContainer" containerID="9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6"
Sep 11 12:19:03.084125 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:19:03.084103    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6\": container with ID starting with 9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6 not found: ID does not exist" containerID="9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6"
Sep 11 12:19:03.084194 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.084135    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6"} err="failed to get container status \"9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6\": rpc error: code = NotFound desc = could not find container \"9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6\": container with ID starting with 9c0c064f461dc6a6d2d3dbfbf794695f86b74a6010ac7428dbff0c6411e079a6 not found: ID does not exist"
Sep 11 12:19:03.084981 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.084965    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"]
Sep 11 12:19:03.088850 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.088825    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-49995f-549d9bc75f-t5s69"]
Sep 11 12:19:03.104030 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.104010    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"]
Sep 11 12:19:03.110002 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:03.109983    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-49995f-5f775df84b-gmghj"]
Sep 11 12:19:04.824774 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:04.824737    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0569cf5f-de74-4245-95f5-9168393902b6" path="/var/lib/kubelet/pods/0569cf5f-de74-4245-95f5-9168393902b6/volumes"
Sep 11 12:19:04.825215 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:04.825103    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="205afe17-9751-462a-8269-7d57e89136ab" path="/var/lib/kubelet/pods/205afe17-9751-462a-8269-7d57e89136ab/volumes"
Sep 11 12:19:06.995476 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:06.995439    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:19:08.606213 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:08.606181    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:08.674041 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:08.674012    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:08.883418 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:08.883326    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:12.331665 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.331626    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"]
Sep 11 12:19:12.332257 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.332233    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="205afe17-9751-462a-8269-7d57e89136ab" containerName="payload-pre-processing"
Sep 11 12:19:12.332320 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.332261    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="205afe17-9751-462a-8269-7d57e89136ab" containerName="payload-pre-processing"
Sep 11 12:19:12.332320 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.332275    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0569cf5f-de74-4245-95f5-9168393902b6" containerName="payload-processing"
Sep 11 12:19:12.332320 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.332281    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0569cf5f-de74-4245-95f5-9168393902b6" containerName="payload-processing"
Sep 11 12:19:12.332428 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.332357    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0569cf5f-de74-4245-95f5-9168393902b6" containerName="payload-processing"
Sep 11 12:19:12.332428 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.332368    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="205afe17-9751-462a-8269-7d57e89136ab" containerName="payload-pre-processing"
Sep 11 12:19:12.337324 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.337304    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"
Sep 11 12:19:12.341697 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.341670    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:19:12.341928 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.341719    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:19:12.342793 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.342749    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"]
Sep 11 12:19:12.397247 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.397223    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0364cd87-76a3-4808-a98b-fbc481d88e10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr\" (UID: \"0364cd87-76a3-4808-a98b-fbc481d88e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"
Sep 11 12:19:12.461164 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.461134    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:12.498384 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.498360    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0364cd87-76a3-4808-a98b-fbc481d88e10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr\" (UID: \"0364cd87-76a3-4808-a98b-fbc481d88e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"
Sep 11 12:19:12.498937 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.498919    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0364cd87-76a3-4808-a98b-fbc481d88e10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr\" (UID: \"0364cd87-76a3-4808-a98b-fbc481d88e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"
Sep 11 12:19:12.651471 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.651411    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"
Sep 11 12:19:12.927723 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:12.927668    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"]
Sep 11 12:19:12.930171 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:19:12.930141    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0364cd87_76a3_4808_a98b_fbc481d88e10.slice/crio-dbce67065ca6e99d2843432bca74a7585704b5d2ee011a5b474112bc9e37120b WatchSource:0}: Error finding container dbce67065ca6e99d2843432bca74a7585704b5d2ee011a5b474112bc9e37120b: Status 404 returned error can't find the container with id dbce67065ca6e99d2843432bca74a7585704b5d2ee011a5b474112bc9e37120b
Sep 11 12:19:13.123443 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:13.123412    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr" event={"ID":"0364cd87-76a3-4808-a98b-fbc481d88e10","Type":"ContainerStarted","Data":"dbce67065ca6e99d2843432bca74a7585704b5d2ee011a5b474112bc9e37120b"}
Sep 11 12:19:14.130937 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:14.130908    2586 generic.go:356] "Generic (PLEG): container finished" podID="0364cd87-76a3-4808-a98b-fbc481d88e10" containerID="8ab3ac319e6408b353ccefdcc8c4a56a492d49a1e4cf36bf9c88e5a74e9dfd98" exitCode=0
Sep 11 12:19:14.131299 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:14.130956    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr" event={"ID":"0364cd87-76a3-4808-a98b-fbc481d88e10","Type":"ContainerDied","Data":"8ab3ac319e6408b353ccefdcc8c4a56a492d49a1e4cf36bf9c88e5a74e9dfd98"}
Sep 11 12:19:15.270159 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:15.270134    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"
Sep 11 12:19:15.321469 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:15.321441    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0364cd87-76a3-4808-a98b-fbc481d88e10-maas-api-service-ca\") pod \"0364cd87-76a3-4808-a98b-fbc481d88e10\" (UID: \"0364cd87-76a3-4808-a98b-fbc481d88e10\") "
Sep 11 12:19:15.321792 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:15.321766    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0364cd87-76a3-4808-a98b-fbc481d88e10-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0364cd87-76a3-4808-a98b-fbc481d88e10" (UID: "0364cd87-76a3-4808-a98b-fbc481d88e10"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:15.422989 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:15.422928    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0364cd87-76a3-4808-a98b-fbc481d88e10-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:16.143263 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:16.143232    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"
Sep 11 12:19:16.143549 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:16.143225    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr" event={"ID":"0364cd87-76a3-4808-a98b-fbc481d88e10","Type":"ContainerDied","Data":"dbce67065ca6e99d2843432bca74a7585704b5d2ee011a5b474112bc9e37120b"}
Sep 11 12:19:16.143549 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:16.143368    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbce67065ca6e99d2843432bca74a7585704b5d2ee011a5b474112bc9e37120b"
Sep 11 12:19:25.704504 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:25.704445    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:19:27.917670 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:27.917628    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:27.938095 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:27.938068    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"]
Sep 11 12:19:27.938305 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:27.938280    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r" podUID="d99da832-a425-4b34-8559-c09ea93d0367" containerName="payload-processing" containerID="cri-o://33d6d67018432ddd4f5c79c4400011dde5e9e2db39ada55b31ad58f503374e84" gracePeriod=30
Sep 11 12:19:27.949535 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:27.949508    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"]
Sep 11 12:19:27.949799 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:27.949775    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh" podUID="56b74704-8198-4785-a39e-430206e1e0a5" containerName="payload-pre-processing" containerID="cri-o://8a2bb431492279565ec6d35957bdba891648c9a929fd253c7305742f2e1e3940" gracePeriod=30
Sep 11 12:19:27.987432 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:27.987392    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:28.088378 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.088350    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:28.113774 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.113744    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"]
Sep 11 12:19:28.119565 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.119506    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9c3012-kcqlr"]
Sep 11 12:19:28.169727 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.169659    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:28.207739 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.207713    2586 generic.go:356] "Generic (PLEG): container finished" podID="d99da832-a425-4b34-8559-c09ea93d0367" containerID="33d6d67018432ddd4f5c79c4400011dde5e9e2db39ada55b31ad58f503374e84" exitCode=0
Sep 11 12:19:28.207861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.207786    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r" event={"ID":"d99da832-a425-4b34-8559-c09ea93d0367","Type":"ContainerDied","Data":"33d6d67018432ddd4f5c79c4400011dde5e9e2db39ada55b31ad58f503374e84"}
Sep 11 12:19:28.207861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.207843    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r" event={"ID":"d99da832-a425-4b34-8559-c09ea93d0367","Type":"ContainerDied","Data":"4a4a4bd4b9250aad2857aca373f2551ba7e78e41492e7ab2af7838a7c4991e99"}
Sep 11 12:19:28.207966 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.207865    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a4a4bd4b9250aad2857aca373f2551ba7e78e41492e7ab2af7838a7c4991e99"
Sep 11 12:19:28.209216 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.209192    2586 generic.go:356] "Generic (PLEG): container finished" podID="56b74704-8198-4785-a39e-430206e1e0a5" containerID="8a2bb431492279565ec6d35957bdba891648c9a929fd253c7305742f2e1e3940" exitCode=0
Sep 11 12:19:28.209331 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.209242    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh" event={"ID":"56b74704-8198-4785-a39e-430206e1e0a5","Type":"ContainerDied","Data":"8a2bb431492279565ec6d35957bdba891648c9a929fd253c7305742f2e1e3940"}
Sep 11 12:19:28.224080 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.224064    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:19:28.227128 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.227107    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:19:28.325876 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.325842    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b74704-8198-4785-a39e-430206e1e0a5-plugins-config-volume\") pod \"56b74704-8198-4785-a39e-430206e1e0a5\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") "
Sep 11 12:19:28.326008 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.325901    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g9wk\" (UniqueName: \"kubernetes.io/projected/56b74704-8198-4785-a39e-430206e1e0a5-kube-api-access-5g9wk\") pod \"56b74704-8198-4785-a39e-430206e1e0a5\" (UID: \"56b74704-8198-4785-a39e-430206e1e0a5\") "
Sep 11 12:19:28.326008 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.325926    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkzj7\" (UniqueName: \"kubernetes.io/projected/d99da832-a425-4b34-8559-c09ea93d0367-kube-api-access-mkzj7\") pod \"d99da832-a425-4b34-8559-c09ea93d0367\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") "
Sep 11 12:19:28.326085 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.326069    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99da832-a425-4b34-8559-c09ea93d0367-plugins-config-volume\") pod \"d99da832-a425-4b34-8559-c09ea93d0367\" (UID: \"d99da832-a425-4b34-8559-c09ea93d0367\") "
Sep 11 12:19:28.326239 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.326219    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56b74704-8198-4785-a39e-430206e1e0a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56b74704-8198-4785-a39e-430206e1e0a5" (UID: "56b74704-8198-4785-a39e-430206e1e0a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:28.326363 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.326348    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b74704-8198-4785-a39e-430206e1e0a5-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.326463 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.326444    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d99da832-a425-4b34-8559-c09ea93d0367-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d99da832-a425-4b34-8559-c09ea93d0367" (UID: "d99da832-a425-4b34-8559-c09ea93d0367"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:28.328088 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.328072    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56b74704-8198-4785-a39e-430206e1e0a5-kube-api-access-5g9wk" (OuterVolumeSpecName: "kube-api-access-5g9wk") pod "56b74704-8198-4785-a39e-430206e1e0a5" (UID: "56b74704-8198-4785-a39e-430206e1e0a5"). InnerVolumeSpecName "kube-api-access-5g9wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:28.328151 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.328103    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99da832-a425-4b34-8559-c09ea93d0367-kube-api-access-mkzj7" (OuterVolumeSpecName: "kube-api-access-mkzj7") pod "d99da832-a425-4b34-8559-c09ea93d0367" (UID: "d99da832-a425-4b34-8559-c09ea93d0367"). InnerVolumeSpecName "kube-api-access-mkzj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:28.427406 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.427358    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d99da832-a425-4b34-8559-c09ea93d0367-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.427406 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.427377    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g9wk\" (UniqueName: \"kubernetes.io/projected/56b74704-8198-4785-a39e-430206e1e0a5-kube-api-access-5g9wk\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.427406 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.427386    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkzj7\" (UniqueName: \"kubernetes.io/projected/d99da832-a425-4b34-8559-c09ea93d0367-kube-api-access-mkzj7\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.460389 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.460355    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:28.824986 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:28.824952    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0364cd87-76a3-4808-a98b-fbc481d88e10" path="/var/lib/kubelet/pods/0364cd87-76a3-4808-a98b-fbc481d88e10/volumes"
Sep 11 12:19:29.215184 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.215115    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"
Sep 11 12:19:29.215184 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.215150    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh" event={"ID":"56b74704-8198-4785-a39e-430206e1e0a5","Type":"ContainerDied","Data":"2bd88b152b56399074086390e3b9c6b8cfe9560272471c8b12005bf0e6de8f29"}
Sep 11 12:19:29.215618 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.215233    2586 scope.go:117] "RemoveContainer" containerID="8a2bb431492279565ec6d35957bdba891648c9a929fd253c7305742f2e1e3940"
Sep 11 12:19:29.215618 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.215328    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"
Sep 11 12:19:29.233980 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.233937    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"]
Sep 11 12:19:29.240038 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.240020    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9c3012-598957f5d6-wn7vh"]
Sep 11 12:19:29.251812 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.251792    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"]
Sep 11 12:19:29.254623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:29.254605    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9c3012-f94b497b5-6dv4r"]
Sep 11 12:19:30.823869 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:30.823829    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56b74704-8198-4785-a39e-430206e1e0a5" path="/var/lib/kubelet/pods/56b74704-8198-4785-a39e-430206e1e0a5/volumes"
Sep 11 12:19:30.824413 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:30.824341    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d99da832-a425-4b34-8559-c09ea93d0367" path="/var/lib/kubelet/pods/d99da832-a425-4b34-8559-c09ea93d0367/volumes"
Sep 11 12:19:33.441300 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:33.441267    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"]
Sep 11 12:19:33.484368 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:33.484340    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:39.100274 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:39.100238    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:19:46.145281 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:46.145236    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:19:48.460602 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.460558    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx" podUID="45bea232-ccc0-4377-9abb-91053ca43096" containerName="main" containerID="cri-o://929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c" gracePeriod=45
Sep 11 12:19:48.709881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.709857    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:19:48.897146 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897114    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-home\") pod \"45bea232-ccc0-4377-9abb-91053ca43096\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") "
Sep 11 12:19:48.897332 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897207    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-dshm\") pod \"45bea232-ccc0-4377-9abb-91053ca43096\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") "
Sep 11 12:19:48.897332 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897245    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szdfx\" (UniqueName: \"kubernetes.io/projected/45bea232-ccc0-4377-9abb-91053ca43096-kube-api-access-szdfx\") pod \"45bea232-ccc0-4377-9abb-91053ca43096\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") "
Sep 11 12:19:48.897332 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897282    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-model-cache\") pod \"45bea232-ccc0-4377-9abb-91053ca43096\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") "
Sep 11 12:19:48.897332 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897318    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-tmp-dir\") pod \"45bea232-ccc0-4377-9abb-91053ca43096\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") "
Sep 11 12:19:48.897613 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897351    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-home" (OuterVolumeSpecName: "home") pod "45bea232-ccc0-4377-9abb-91053ca43096" (UID: "45bea232-ccc0-4377-9abb-91053ca43096"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:48.897613 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897383    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bea232-ccc0-4377-9abb-91053ca43096-tls-certs\") pod \"45bea232-ccc0-4377-9abb-91053ca43096\" (UID: \"45bea232-ccc0-4377-9abb-91053ca43096\") "
Sep 11 12:19:48.897613 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897573    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-model-cache" (OuterVolumeSpecName: "model-cache") pod "45bea232-ccc0-4377-9abb-91053ca43096" (UID: "45bea232-ccc0-4377-9abb-91053ca43096"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:48.897779 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897688    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "45bea232-ccc0-4377-9abb-91053ca43096" (UID: "45bea232-ccc0-4377-9abb-91053ca43096"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:48.897838 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897799    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:48.897838 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897819    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:48.897838 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.897830    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:48.899717 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.899696    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45bea232-ccc0-4377-9abb-91053ca43096-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "45bea232-ccc0-4377-9abb-91053ca43096" (UID: "45bea232-ccc0-4377-9abb-91053ca43096"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:19:48.899839 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.899726    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-dshm" (OuterVolumeSpecName: "dshm") pod "45bea232-ccc0-4377-9abb-91053ca43096" (UID: "45bea232-ccc0-4377-9abb-91053ca43096"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:19:48.899839 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.899776    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45bea232-ccc0-4377-9abb-91053ca43096-kube-api-access-szdfx" (OuterVolumeSpecName: "kube-api-access-szdfx") pod "45bea232-ccc0-4377-9abb-91053ca43096" (UID: "45bea232-ccc0-4377-9abb-91053ca43096"). InnerVolumeSpecName "kube-api-access-szdfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:48.998503 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.998453    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bea232-ccc0-4377-9abb-91053ca43096-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:48.998503 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.998473    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bea232-ccc0-4377-9abb-91053ca43096-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:48.998503 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:48.998500    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szdfx\" (UniqueName: \"kubernetes.io/projected/45bea232-ccc0-4377-9abb-91053ca43096-kube-api-access-szdfx\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:19:49.321134 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.321099    2586 generic.go:356] "Generic (PLEG): container finished" podID="45bea232-ccc0-4377-9abb-91053ca43096" containerID="929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c" exitCode=0
Sep 11 12:19:49.321276 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.321152    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"
Sep 11 12:19:49.321276 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.321191    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx" event={"ID":"45bea232-ccc0-4377-9abb-91053ca43096","Type":"ContainerDied","Data":"929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c"}
Sep 11 12:19:49.321276 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.321237    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx" event={"ID":"45bea232-ccc0-4377-9abb-91053ca43096","Type":"ContainerDied","Data":"fec963ab501ce0c94599d0455c7f5c49d1ceb6d9d39f16d2446e0a4d86f7fd47"}
Sep 11 12:19:49.321276 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.321255    2586 scope.go:117] "RemoveContainer" containerID="929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c"
Sep 11 12:19:49.331120 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.331101    2586 scope.go:117] "RemoveContainer" containerID="929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c"
Sep 11 12:19:49.331368 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:19:49.331349    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c\": container with ID starting with 929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c not found: ID does not exist" containerID="929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c"
Sep 11 12:19:49.331416 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.331376    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c"} err="failed to get container status \"929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c\": rpc error: code = NotFound desc = could not find container \"929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c\": container with ID starting with 929dc632862e4199c300c522c4997b48b54582952ca269c24aa4ae3d5002825c not found: ID does not exist"
Sep 11 12:19:49.342908 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.342882    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"]
Sep 11 12:19:49.347906 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:49.347884    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-9c3012/ipp-route-9c3012-kserve-6c596fc87-f98xx"]
Sep 11 12:19:50.824357 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:19:50.824309    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45bea232-ccc0-4377-9abb-91053ca43096" path="/var/lib/kubelet/pods/45bea232-ccc0-4377-9abb-91053ca43096/volumes"
Sep 11 12:20:09.762675 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:09.762588    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:10.471290 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:10.471230    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:15.974436 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:15.974392    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:19.994727 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:19.994678    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:21.631504 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:21.631453    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:21.684926 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:21.684897    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:21.916836 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:21.916762    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:22.286544 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286455    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"]
Sep 11 12:20:22.286893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286874    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56b74704-8198-4785-a39e-430206e1e0a5" containerName="payload-pre-processing"
Sep 11 12:20:22.286893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286893    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="56b74704-8198-4785-a39e-430206e1e0a5" containerName="payload-pre-processing"
Sep 11 12:20:22.287049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286911    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45bea232-ccc0-4377-9abb-91053ca43096" containerName="main"
Sep 11 12:20:22.287049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286921    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="45bea232-ccc0-4377-9abb-91053ca43096" containerName="main"
Sep 11 12:20:22.287049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286960    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d99da832-a425-4b34-8559-c09ea93d0367" containerName="payload-processing"
Sep 11 12:20:22.287049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286968    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="d99da832-a425-4b34-8559-c09ea93d0367" containerName="payload-processing"
Sep 11 12:20:22.287049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286977    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0364cd87-76a3-4808-a98b-fbc481d88e10" containerName="revoke-keys"
Sep 11 12:20:22.287049 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.286984    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0364cd87-76a3-4808-a98b-fbc481d88e10" containerName="revoke-keys"
Sep 11 12:20:22.287282 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.287071    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="45bea232-ccc0-4377-9abb-91053ca43096" containerName="main"
Sep 11 12:20:22.287282 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.287079    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="d99da832-a425-4b34-8559-c09ea93d0367" containerName="payload-processing"
Sep 11 12:20:22.287282 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.287087    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0364cd87-76a3-4808-a98b-fbc481d88e10" containerName="revoke-keys"
Sep 11 12:20:22.287282 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.287093    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="56b74704-8198-4785-a39e-430206e1e0a5" containerName="payload-pre-processing"
Sep 11 12:20:22.289954 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.289932    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"
Sep 11 12:20:22.292726 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.292704    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:20:22.292859 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.292737    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:20:22.298721 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.298698    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"]
Sep 11 12:20:22.358878 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.358846    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c8364-56b0-4431-90a5-1b824489b7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc\" (UID: \"e04c8364-56b0-4431-90a5-1b824489b7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"
Sep 11 12:20:22.392298 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.392269    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:22.459496 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.459458    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c8364-56b0-4431-90a5-1b824489b7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc\" (UID: \"e04c8364-56b0-4431-90a5-1b824489b7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"
Sep 11 12:20:22.460065 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.460044    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c8364-56b0-4431-90a5-1b824489b7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc\" (UID: \"e04c8364-56b0-4431-90a5-1b824489b7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"
Sep 11 12:20:22.602872 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.602850    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"
Sep 11 12:20:22.872651 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:22.872581    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"]
Sep 11 12:20:22.874988 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:20:22.874955    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04c8364_56b0_4431_90a5_1b824489b7dc.slice/crio-785adc1b4d0d639a64dc180524af736c1ee3607f68f66d9e5101c0fd8936c82a WatchSource:0}: Error finding container 785adc1b4d0d639a64dc180524af736c1ee3607f68f66d9e5101c0fd8936c82a: Status 404 returned error can't find the container with id 785adc1b4d0d639a64dc180524af736c1ee3607f68f66d9e5101c0fd8936c82a
Sep 11 12:20:23.512110 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:23.512031    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc" event={"ID":"e04c8364-56b0-4431-90a5-1b824489b7dc","Type":"ContainerStarted","Data":"75c8b2d758439a591eaced0f73b7713a4f98f9c72eb99b961b586f860b569930"}
Sep 11 12:20:23.512110 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:23.512081    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc" event={"ID":"e04c8364-56b0-4431-90a5-1b824489b7dc","Type":"ContainerStarted","Data":"785adc1b4d0d639a64dc180524af736c1ee3607f68f66d9e5101c0fd8936c82a"}
Sep 11 12:20:23.531217 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:23.531169    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc" podStartSLOduration=1.531154425 podStartE2EDuration="1.531154425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:20:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:20:23.529526378 +0000 UTC m=+1717.240197716" watchObservedRunningTime="2026-09-11 12:20:23.531154425 +0000 UTC m=+1717.241825753"
Sep 11 12:20:24.520583 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:24.520538    2586 generic.go:356] "Generic (PLEG): container finished" podID="e04c8364-56b0-4431-90a5-1b824489b7dc" containerID="75c8b2d758439a591eaced0f73b7713a4f98f9c72eb99b961b586f860b569930" exitCode=0
Sep 11 12:20:24.520993 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:24.520602    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc" event={"ID":"e04c8364-56b0-4431-90a5-1b824489b7dc","Type":"ContainerDied","Data":"75c8b2d758439a591eaced0f73b7713a4f98f9c72eb99b961b586f860b569930"}
Sep 11 12:20:25.666028 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:25.666006    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"
Sep 11 12:20:25.794371 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:25.794343    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c8364-56b0-4431-90a5-1b824489b7dc-maas-api-service-ca\") pod \"e04c8364-56b0-4431-90a5-1b824489b7dc\" (UID: \"e04c8364-56b0-4431-90a5-1b824489b7dc\") "
Sep 11 12:20:25.794683 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:25.794662    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e04c8364-56b0-4431-90a5-1b824489b7dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e04c8364-56b0-4431-90a5-1b824489b7dc" (UID: "e04c8364-56b0-4431-90a5-1b824489b7dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:25.895823 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:25.895799    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c8364-56b0-4431-90a5-1b824489b7dc-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:26.532914 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:26.532888    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"
Sep 11 12:20:26.532914 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:26.532901    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc" event={"ID":"e04c8364-56b0-4431-90a5-1b824489b7dc","Type":"ContainerDied","Data":"785adc1b4d0d639a64dc180524af736c1ee3607f68f66d9e5101c0fd8936c82a"}
Sep 11 12:20:26.533102 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:26.532948    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="785adc1b4d0d639a64dc180524af736c1ee3607f68f66d9e5101c0fd8936c82a"
Sep 11 12:20:32.772217 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.772180    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:32.791632 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.791603    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"]
Sep 11 12:20:32.791887 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.791841    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t" podUID="4f6e3590-c248-4437-814d-5ec4853ac591" containerName="payload-processing" containerID="cri-o://6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88" gracePeriod=30
Sep 11 12:20:32.800694 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.800417    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"]
Sep 11 12:20:32.801510 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.801455    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px" podUID="e6a0655b-4811-4541-9885-7aa8c28e3b33" containerName="payload-pre-processing" containerID="cri-o://70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89" gracePeriod=30
Sep 11 12:20:32.849628 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.849597    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:32.948916 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.948146    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:32.954978 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.954950    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"]
Sep 11 12:20:32.957302 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:32.957276    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6d324e-x9zwc"]
Sep 11 12:20:33.050852 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.050828    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:20:33.080047 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.080027    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:20:33.154204 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.154182    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6e3590-c248-4437-814d-5ec4853ac591-plugins-config-volume\") pod \"4f6e3590-c248-4437-814d-5ec4853ac591\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") "
Sep 11 12:20:33.154347 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.154223    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0655b-4811-4541-9885-7aa8c28e3b33-plugins-config-volume\") pod \"e6a0655b-4811-4541-9885-7aa8c28e3b33\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") "
Sep 11 12:20:33.154347 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.154288    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9227\" (UniqueName: \"kubernetes.io/projected/e6a0655b-4811-4541-9885-7aa8c28e3b33-kube-api-access-t9227\") pod \"e6a0655b-4811-4541-9885-7aa8c28e3b33\" (UID: \"e6a0655b-4811-4541-9885-7aa8c28e3b33\") "
Sep 11 12:20:33.154347 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.154330    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82bmv\" (UniqueName: \"kubernetes.io/projected/4f6e3590-c248-4437-814d-5ec4853ac591-kube-api-access-82bmv\") pod \"4f6e3590-c248-4437-814d-5ec4853ac591\" (UID: \"4f6e3590-c248-4437-814d-5ec4853ac591\") "
Sep 11 12:20:33.154625 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.154600    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f6e3590-c248-4437-814d-5ec4853ac591-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f6e3590-c248-4437-814d-5ec4853ac591" (UID: "4f6e3590-c248-4437-814d-5ec4853ac591"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:33.154625 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.154610    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6a0655b-4811-4541-9885-7aa8c28e3b33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6a0655b-4811-4541-9885-7aa8c28e3b33" (UID: "e6a0655b-4811-4541-9885-7aa8c28e3b33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:33.156594 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.156565    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f6e3590-c248-4437-814d-5ec4853ac591-kube-api-access-82bmv" (OuterVolumeSpecName: "kube-api-access-82bmv") pod "4f6e3590-c248-4437-814d-5ec4853ac591" (UID: "4f6e3590-c248-4437-814d-5ec4853ac591"). InnerVolumeSpecName "kube-api-access-82bmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:20:33.156696 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.156678    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a0655b-4811-4541-9885-7aa8c28e3b33-kube-api-access-t9227" (OuterVolumeSpecName: "kube-api-access-t9227") pod "e6a0655b-4811-4541-9885-7aa8c28e3b33" (UID: "e6a0655b-4811-4541-9885-7aa8c28e3b33"). InnerVolumeSpecName "kube-api-access-t9227". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:20:33.255128 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.255106    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0655b-4811-4541-9885-7aa8c28e3b33-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.255128 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.255125    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9227\" (UniqueName: \"kubernetes.io/projected/e6a0655b-4811-4541-9885-7aa8c28e3b33-kube-api-access-t9227\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.255257 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.255134    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82bmv\" (UniqueName: \"kubernetes.io/projected/4f6e3590-c248-4437-814d-5ec4853ac591-kube-api-access-82bmv\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.255257 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.255145    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6e3590-c248-4437-814d-5ec4853ac591-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.291352 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.291328    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:33.395928 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.395896    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:33.571415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.571341    2586 generic.go:356] "Generic (PLEG): container finished" podID="4f6e3590-c248-4437-814d-5ec4853ac591" containerID="6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88" exitCode=0
Sep 11 12:20:33.571415 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.571394    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"
Sep 11 12:20:33.571630 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.571434    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t" event={"ID":"4f6e3590-c248-4437-814d-5ec4853ac591","Type":"ContainerDied","Data":"6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88"}
Sep 11 12:20:33.571630 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.571516    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t" event={"ID":"4f6e3590-c248-4437-814d-5ec4853ac591","Type":"ContainerDied","Data":"e34ae3990b395d8dfcfa4ebb66ef872088c5fc33d43ea4ae4912ee85ca0500e8"}
Sep 11 12:20:33.571630 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.571544    2586 scope.go:117] "RemoveContainer" containerID="6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88"
Sep 11 12:20:33.573073 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.573045    2586 generic.go:356] "Generic (PLEG): container finished" podID="e6a0655b-4811-4541-9885-7aa8c28e3b33" containerID="70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89" exitCode=0
Sep 11 12:20:33.573180 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.573091    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px" event={"ID":"e6a0655b-4811-4541-9885-7aa8c28e3b33","Type":"ContainerDied","Data":"70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89"}
Sep 11 12:20:33.573180 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.573111    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"
Sep 11 12:20:33.573180 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.573131    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px" event={"ID":"e6a0655b-4811-4541-9885-7aa8c28e3b33","Type":"ContainerDied","Data":"0db53d398e642526a4359a7fe3f3ce5e5ef21375d26234d865a0954e604f569b"}
Sep 11 12:20:33.586931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.584217    2586 scope.go:117] "RemoveContainer" containerID="6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88"
Sep 11 12:20:33.586931 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:20:33.584680    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88\": container with ID starting with 6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88 not found: ID does not exist" containerID="6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88"
Sep 11 12:20:33.586931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.584718    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88"} err="failed to get container status \"6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88\": rpc error: code = NotFound desc = could not find container \"6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88\": container with ID starting with 6a1df0b69a536b4c97314cac2194821c3efd23dd0abf57c9cd52967b6a37bf88 not found: ID does not exist"
Sep 11 12:20:33.586931 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.584746    2586 scope.go:117] "RemoveContainer" containerID="70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89"
Sep 11 12:20:33.595391 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.595374    2586 scope.go:117] "RemoveContainer" containerID="70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89"
Sep 11 12:20:33.595780 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:20:33.595760    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89\": container with ID starting with 70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89 not found: ID does not exist" containerID="70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89"
Sep 11 12:20:33.595868 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.595788    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89"} err="failed to get container status \"70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89\": rpc error: code = NotFound desc = could not find container \"70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89\": container with ID starting with 70960af4b57a60380485aada04f277acdda60451bd90357ffbe7943b009dfa89 not found: ID does not exist"
Sep 11 12:20:33.602763 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.602738    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"]
Sep 11 12:20:33.606449 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.606427    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d324e-8657bcfd55-df25t"]
Sep 11 12:20:33.617129 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.617109    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"]
Sep 11 12:20:33.620291 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:33.620271    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d324e-667cb8b546-z86px"]
Sep 11 12:20:34.823727 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:34.823693    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f6e3590-c248-4437-814d-5ec4853ac591" path="/var/lib/kubelet/pods/4f6e3590-c248-4437-814d-5ec4853ac591/volumes"
Sep 11 12:20:34.824087 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:34.823991    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e04c8364-56b0-4431-90a5-1b824489b7dc" path="/var/lib/kubelet/pods/e04c8364-56b0-4431-90a5-1b824489b7dc/volumes"
Sep 11 12:20:34.824292 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:34.824278    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a0655b-4811-4541-9885-7aa8c28e3b33" path="/var/lib/kubelet/pods/e6a0655b-4811-4541-9885-7aa8c28e3b33/volumes"
Sep 11 12:20:36.195442 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:36.195401    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:20:39.222260 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.222225    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:39.281753 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.281727    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:39.474745 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.474673    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:39.812548 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.812517    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"]
Sep 11 12:20:39.813003 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.812984    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a0655b-4811-4541-9885-7aa8c28e3b33" containerName="payload-pre-processing"
Sep 11 12:20:39.813003 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813006    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a0655b-4811-4541-9885-7aa8c28e3b33" containerName="payload-pre-processing"
Sep 11 12:20:39.813003 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813022    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e04c8364-56b0-4431-90a5-1b824489b7dc" containerName="revoke-keys"
Sep 11 12:20:39.813003 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813030    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e04c8364-56b0-4431-90a5-1b824489b7dc" containerName="revoke-keys"
Sep 11 12:20:39.813306 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813073    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f6e3590-c248-4437-814d-5ec4853ac591" containerName="payload-processing"
Sep 11 12:20:39.813306 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813082    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f6e3590-c248-4437-814d-5ec4853ac591" containerName="payload-processing"
Sep 11 12:20:39.813306 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813197    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e04c8364-56b0-4431-90a5-1b824489b7dc" containerName="revoke-keys"
Sep 11 12:20:39.813306 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813208    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f6e3590-c248-4437-814d-5ec4853ac591" containerName="payload-processing"
Sep 11 12:20:39.813306 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.813221    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a0655b-4811-4541-9885-7aa8c28e3b33" containerName="payload-pre-processing"
Sep 11 12:20:39.817837 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.817817    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"
Sep 11 12:20:39.822412 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.822380    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:20:39.822689 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.822667    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:20:39.823428 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.823400    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"]
Sep 11 12:20:39.908920 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:39.908893    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28458843-b699-4c01-a021-eeda83938547-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9\" (UID: \"28458843-b699-4c01-a021-eeda83938547\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"
Sep 11 12:20:40.010059 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:40.010026    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28458843-b699-4c01-a021-eeda83938547-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9\" (UID: \"28458843-b699-4c01-a021-eeda83938547\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"
Sep 11 12:20:40.010603 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:40.010585    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28458843-b699-4c01-a021-eeda83938547-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9\" (UID: \"28458843-b699-4c01-a021-eeda83938547\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"
Sep 11 12:20:40.058878 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:40.058849    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:40.129851 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:40.129796    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"
Sep 11 12:20:40.405114 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:40.405089    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"]
Sep 11 12:20:40.407313 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:20:40.407284    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28458843_b699_4c01_a021_eeda83938547.slice/crio-a08bbabfdee593103bf4538acf265da3e84c61184c43eb6bbbc86e7602999370 WatchSource:0}: Error finding container a08bbabfdee593103bf4538acf265da3e84c61184c43eb6bbbc86e7602999370: Status 404 returned error can't find the container with id a08bbabfdee593103bf4538acf265da3e84c61184c43eb6bbbc86e7602999370
Sep 11 12:20:40.611469 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:40.611438    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9" event={"ID":"28458843-b699-4c01-a021-eeda83938547","Type":"ContainerStarted","Data":"a08bbabfdee593103bf4538acf265da3e84c61184c43eb6bbbc86e7602999370"}
Sep 11 12:20:41.618582 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:41.618550    2586 generic.go:356] "Generic (PLEG): container finished" podID="28458843-b699-4c01-a021-eeda83938547" containerID="5e0e324c8eb49ff4c3ca6e81247c51887f67501401a7b80bcb7e33a50983accb" exitCode=0
Sep 11 12:20:41.618925 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:41.618633    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9" event={"ID":"28458843-b699-4c01-a021-eeda83938547","Type":"ContainerDied","Data":"5e0e324c8eb49ff4c3ca6e81247c51887f67501401a7b80bcb7e33a50983accb"}
Sep 11 12:20:42.757785 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:42.757760    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"
Sep 11 12:20:42.933767 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:42.933710    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28458843-b699-4c01-a021-eeda83938547-maas-api-service-ca\") pod \"28458843-b699-4c01-a021-eeda83938547\" (UID: \"28458843-b699-4c01-a021-eeda83938547\") "
Sep 11 12:20:42.934031 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:42.934008    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28458843-b699-4c01-a021-eeda83938547-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28458843-b699-4c01-a021-eeda83938547" (UID: "28458843-b699-4c01-a021-eeda83938547"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:43.034849 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:43.034824    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28458843-b699-4c01-a021-eeda83938547-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:43.632905 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:43.632874    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"
Sep 11 12:20:43.633045 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:43.632913    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9" event={"ID":"28458843-b699-4c01-a021-eeda83938547","Type":"ContainerDied","Data":"a08bbabfdee593103bf4538acf265da3e84c61184c43eb6bbbc86e7602999370"}
Sep 11 12:20:43.633045 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:43.632962    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a08bbabfdee593103bf4538acf265da3e84c61184c43eb6bbbc86e7602999370"
Sep 11 12:20:50.385920 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.385882    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:50.418452 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.418421    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"]
Sep 11 12:20:50.418786 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.418758    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx" podUID="0ae29249-d71a-43a8-b24f-7de33b20e2f5" containerName="payload-pre-processing" containerID="cri-o://ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf" gracePeriod=30
Sep 11 12:20:50.482802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.482766    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:50.575399 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.575370    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"]
Sep 11 12:20:50.579174 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.579143    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:50.579611 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.579589    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-88968d-n8sb9"]
Sep 11 12:20:50.630603 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.630574    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:50.666781 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.666762    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:20:50.675171 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.675144    2586 generic.go:356] "Generic (PLEG): container finished" podID="0ae29249-d71a-43a8-b24f-7de33b20e2f5" containerID="ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf" exitCode=0
Sep 11 12:20:50.675267 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.675176    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"
Sep 11 12:20:50.675267 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.675186    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx" event={"ID":"0ae29249-d71a-43a8-b24f-7de33b20e2f5","Type":"ContainerDied","Data":"ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf"}
Sep 11 12:20:50.675267 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.675236    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx" event={"ID":"0ae29249-d71a-43a8-b24f-7de33b20e2f5","Type":"ContainerDied","Data":"0eae315687e0a695419ab6f6a11ef5bead796de16ec508f05d6cb7e6853cfc42"}
Sep 11 12:20:50.675267 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.675266    2586 scope.go:117] "RemoveContainer" containerID="ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf"
Sep 11 12:20:50.683674 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.683658    2586 scope.go:117] "RemoveContainer" containerID="ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf"
Sep 11 12:20:50.683946 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:20:50.683928    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf\": container with ID starting with ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf not found: ID does not exist" containerID="ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf"
Sep 11 12:20:50.684009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.683958    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf"} err="failed to get container status \"ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf\": rpc error: code = NotFound desc = could not find container \"ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf\": container with ID starting with ef40ec6842cbb6150e51ec53098c0cdadee90362b90780cc5c9a17e8069e37bf not found: ID does not exist"
Sep 11 12:20:50.800569 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.800544    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rczpl\" (UniqueName: \"kubernetes.io/projected/0ae29249-d71a-43a8-b24f-7de33b20e2f5-kube-api-access-rczpl\") pod \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") "
Sep 11 12:20:50.800716 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.800607    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae29249-d71a-43a8-b24f-7de33b20e2f5-plugins-config-volume\") pod \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\" (UID: \"0ae29249-d71a-43a8-b24f-7de33b20e2f5\") "
Sep 11 12:20:50.800995 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.800967    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae29249-d71a-43a8-b24f-7de33b20e2f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ae29249-d71a-43a8-b24f-7de33b20e2f5" (UID: "0ae29249-d71a-43a8-b24f-7de33b20e2f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:50.802767 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.802747    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae29249-d71a-43a8-b24f-7de33b20e2f5-kube-api-access-rczpl" (OuterVolumeSpecName: "kube-api-access-rczpl") pod "0ae29249-d71a-43a8-b24f-7de33b20e2f5" (UID: "0ae29249-d71a-43a8-b24f-7de33b20e2f5"). InnerVolumeSpecName "kube-api-access-rczpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:20:50.823135 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.823114    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28458843-b699-4c01-a021-eeda83938547" path="/var/lib/kubelet/pods/28458843-b699-4c01-a021-eeda83938547/volumes"
Sep 11 12:20:50.901782 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.901736    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae29249-d71a-43a8-b24f-7de33b20e2f5-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.901782 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.901756    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rczpl\" (UniqueName: \"kubernetes.io/projected/0ae29249-d71a-43a8-b24f-7de33b20e2f5-kube-api-access-rczpl\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.956602 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.956578    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:50.990985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:50.990962    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"]
Sep 11 12:20:51.000832 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:51.000802    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-88968d-846db47874-scqrx"]
Sep 11 12:20:52.823328 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:52.823292    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae29249-d71a-43a8-b24f-7de33b20e2f5" path="/var/lib/kubelet/pods/0ae29249-d71a-43a8-b24f-7de33b20e2f5/volumes"
Sep 11 12:20:53.395216 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:53.395179    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:20:56.943622 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:56.943587    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:57.044009 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.043981    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:57.229929 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.229836    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:57.551518 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.551476    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"]
Sep 11 12:20:57.551815 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.551804    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28458843-b699-4c01-a021-eeda83938547" containerName="revoke-keys"
Sep 11 12:20:57.551861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.551818    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="28458843-b699-4c01-a021-eeda83938547" containerName="revoke-keys"
Sep 11 12:20:57.551861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.551830    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae29249-d71a-43a8-b24f-7de33b20e2f5" containerName="payload-pre-processing"
Sep 11 12:20:57.551861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.551835    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae29249-d71a-43a8-b24f-7de33b20e2f5" containerName="payload-pre-processing"
Sep 11 12:20:57.551955 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.551906    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae29249-d71a-43a8-b24f-7de33b20e2f5" containerName="payload-pre-processing"
Sep 11 12:20:57.551955 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.551914    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="28458843-b699-4c01-a021-eeda83938547" containerName="revoke-keys"
Sep 11 12:20:57.556307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.556289    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"
Sep 11 12:20:57.559317 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.559296    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:20:57.559439 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.559368    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95fz\""
Sep 11 12:20:57.564061 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.563657    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"]
Sep 11 12:20:57.650854 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.650826    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795db35f-0c47-4923-9f75-ec435f5ff8a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g\" (UID: \"795db35f-0c47-4923-9f75-ec435f5ff8a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"
Sep 11 12:20:57.651900 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.651871    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:57.751672 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.751647    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795db35f-0c47-4923-9f75-ec435f5ff8a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g\" (UID: \"795db35f-0c47-4923-9f75-ec435f5ff8a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"
Sep 11 12:20:57.752192 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.752175    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795db35f-0c47-4923-9f75-ec435f5ff8a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g\" (UID: \"795db35f-0c47-4923-9f75-ec435f5ff8a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"
Sep 11 12:20:57.868456 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:57.868392    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"
Sep 11 12:20:58.138179 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:58.138153    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"]
Sep 11 12:20:58.140151 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:20:58.140124    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795db35f_0c47_4923_9f75_ec435f5ff8a5.slice/crio-e70b26abf147b082cb4917fc85be59349fce8dc9a6304ce7840ba97b391816ca WatchSource:0}: Error finding container e70b26abf147b082cb4917fc85be59349fce8dc9a6304ce7840ba97b391816ca: Status 404 returned error can't find the container with id e70b26abf147b082cb4917fc85be59349fce8dc9a6304ce7840ba97b391816ca
Sep 11 12:20:58.722323 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:58.722296    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g" event={"ID":"795db35f-0c47-4923-9f75-ec435f5ff8a5","Type":"ContainerStarted","Data":"e28a11c87e8a223ae1dae2664d98181e3bfb7d06e6d5b914d060c6bc0b5998c6"}
Sep 11 12:20:58.722506 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:58.722348    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g" event={"ID":"795db35f-0c47-4923-9f75-ec435f5ff8a5","Type":"ContainerStarted","Data":"e70b26abf147b082cb4917fc85be59349fce8dc9a6304ce7840ba97b391816ca"}
Sep 11 12:20:58.738907 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:58.738864    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g" podStartSLOduration=1.738850637 podStartE2EDuration="1.738850637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:20:58.736557049 +0000 UTC m=+1752.447228385" watchObservedRunningTime="2026-09-11 12:20:58.738850637 +0000 UTC m=+1752.449521975"
Sep 11 12:20:59.664339 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:59.664302    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:20:59.729430 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:59.729391    2586 generic.go:356] "Generic (PLEG): container finished" podID="795db35f-0c47-4923-9f75-ec435f5ff8a5" containerID="e28a11c87e8a223ae1dae2664d98181e3bfb7d06e6d5b914d060c6bc0b5998c6" exitCode=0
Sep 11 12:20:59.729624 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:20:59.729444    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g" event={"ID":"795db35f-0c47-4923-9f75-ec435f5ff8a5","Type":"ContainerDied","Data":"e28a11c87e8a223ae1dae2664d98181e3bfb7d06e6d5b914d060c6bc0b5998c6"}
Sep 11 12:21:00.868662 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:00.868636    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"
Sep 11 12:21:00.975957 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:00.975931    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795db35f-0c47-4923-9f75-ec435f5ff8a5-maas-api-service-ca\") pod \"795db35f-0c47-4923-9f75-ec435f5ff8a5\" (UID: \"795db35f-0c47-4923-9f75-ec435f5ff8a5\") "
Sep 11 12:21:00.976272 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:00.976254    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/795db35f-0c47-4923-9f75-ec435f5ff8a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "795db35f-0c47-4923-9f75-ec435f5ff8a5" (UID: "795db35f-0c47-4923-9f75-ec435f5ff8a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:21:01.083373 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:01.077587    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795db35f-0c47-4923-9f75-ec435f5ff8a5-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:21:01.741353 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:01.741312    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"
Sep 11 12:21:01.741542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:01.741360    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g" event={"ID":"795db35f-0c47-4923-9f75-ec435f5ff8a5","Type":"ContainerDied","Data":"e70b26abf147b082cb4917fc85be59349fce8dc9a6304ce7840ba97b391816ca"}
Sep 11 12:21:01.741542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:01.741443    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e70b26abf147b082cb4917fc85be59349fce8dc9a6304ce7840ba97b391816ca"
Sep 11 12:21:07.687252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:07.687219    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:08.275970 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.275932    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:08.310340 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.310314    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"]
Sep 11 12:21:08.310594 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.310563    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r" podUID="c586c28c-3d48-4e84-8e10-6148f6e7f84e" containerName="payload-pre-processing" containerID="cri-o://5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f" gracePeriod=30
Sep 11 12:21:08.374914 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.374880    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:08.474082 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.474051    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:08.511405 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.511379    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"]
Sep 11 12:21:08.514428 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.514398    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4d3f83-m5j9g"]
Sep 11 12:21:08.532662 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.532601    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:08.567929 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.567893    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:21:08.629753 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.629725    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c586c28c-3d48-4e84-8e10-6148f6e7f84e-plugins-config-volume\") pod \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") "
Sep 11 12:21:08.629874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.629757    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7bg4\" (UniqueName: \"kubernetes.io/projected/c586c28c-3d48-4e84-8e10-6148f6e7f84e-kube-api-access-z7bg4\") pod \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\" (UID: \"c586c28c-3d48-4e84-8e10-6148f6e7f84e\") "
Sep 11 12:21:08.630083 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.630060    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c586c28c-3d48-4e84-8e10-6148f6e7f84e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c586c28c-3d48-4e84-8e10-6148f6e7f84e" (UID: "c586c28c-3d48-4e84-8e10-6148f6e7f84e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:21:08.631935 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.631914    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c586c28c-3d48-4e84-8e10-6148f6e7f84e-kube-api-access-z7bg4" (OuterVolumeSpecName: "kube-api-access-z7bg4") pod "c586c28c-3d48-4e84-8e10-6148f6e7f84e" (UID: "c586c28c-3d48-4e84-8e10-6148f6e7f84e"). InnerVolumeSpecName "kube-api-access-z7bg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:21:08.730351 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.730318    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c586c28c-3d48-4e84-8e10-6148f6e7f84e-plugins-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:21:08.730351 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.730349    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7bg4\" (UniqueName: \"kubernetes.io/projected/c586c28c-3d48-4e84-8e10-6148f6e7f84e-kube-api-access-z7bg4\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:21:08.781788 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.781756    2586 generic.go:356] "Generic (PLEG): container finished" podID="c586c28c-3d48-4e84-8e10-6148f6e7f84e" containerID="5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f" exitCode=0
Sep 11 12:21:08.781914 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.781801    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"
Sep 11 12:21:08.781914 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.781843    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r" event={"ID":"c586c28c-3d48-4e84-8e10-6148f6e7f84e","Type":"ContainerDied","Data":"5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f"}
Sep 11 12:21:08.781914 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.781893    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r" event={"ID":"c586c28c-3d48-4e84-8e10-6148f6e7f84e","Type":"ContainerDied","Data":"68b3f6c4bbdcf2877bb17a3884d66f1da2bbea8bd542a3e4f6da4c2096bcc157"}
Sep 11 12:21:08.782065 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.781915    2586 scope.go:117] "RemoveContainer" containerID="5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f"
Sep 11 12:21:08.791446 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.791427    2586 scope.go:117] "RemoveContainer" containerID="5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f"
Sep 11 12:21:08.791735 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:21:08.791718    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f\": container with ID starting with 5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f not found: ID does not exist" containerID="5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f"
Sep 11 12:21:08.791802 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.791742    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f"} err="failed to get container status \"5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f\": rpc error: code = NotFound desc = could not find container \"5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f\": container with ID starting with 5f45df5bd1938550c8c7fb849b0f7959a53d3ca47387ee6c80d1a6bee3615d2f not found: ID does not exist"
Sep 11 12:21:08.804116 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.804086    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"]
Sep 11 12:21:08.809167 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.809146    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4d3f83-58b888db86-p896r"]
Sep 11 12:21:08.823343 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.823319    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="795db35f-0c47-4923-9f75-ec435f5ff8a5" path="/var/lib/kubelet/pods/795db35f-0c47-4923-9f75-ec435f5ff8a5/volumes"
Sep 11 12:21:08.823723 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.823710    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c586c28c-3d48-4e84-8e10-6148f6e7f84e" path="/var/lib/kubelet/pods/c586c28c-3d48-4e84-8e10-6148f6e7f84e/volumes"
Sep 11 12:21:08.873936 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:08.873910    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:10.994067 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:10.994025    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:21:14.688295 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:14.688254    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:14.759601 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:14.759563    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:14.987174 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:14.987075    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:20.201294 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:20.201260    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:21:44.597696 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:44.597609    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:44.698703 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:44.698671    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:21:46.902174 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:46.902142    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:21:46.909961 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:46.909937    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:21:47.131158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:47.131127    2586 scope.go:117] "RemoveContainer" containerID="6f00734c7343634c38e25b4c7971841d483a4be45467a1d8d9625d38f351dc76"
Sep 11 12:21:47.139050 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:47.139031    2586 scope.go:117] "RemoveContainer" containerID="ee733455420944e0bf9fd3b482ff4d97a9cdb7dffd40ce8990ace8acca4e5ab3"
Sep 11 12:21:47.146527 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:47.146508    2586 scope.go:117] "RemoveContainer" containerID="23ece39c2868499448f9ee19a3049c623be47e1bf4bd448a352c237b8786665d"
Sep 11 12:21:47.153604 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:21:47.153586    2586 scope.go:117] "RemoveContainer" containerID="f60e6e3569b4f430b33548e700dbaecf7007ce433a50e91d1dc4ecd90bcfb246"
Sep 11 12:22:39.178472 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:22:39.178436    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:40.398285 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:22:40.398247    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:22:41.690733 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:22:41.690668    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:43.200607 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:22:43.200569    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:22:47.191398 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:22:47.191360    2586 scope.go:117] "RemoveContainer" containerID="f7c318cc523b9717ba5c440dff768c98aadce83f8955bbf31b15560508c79ee1"
Sep 11 12:22:52.018199 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:22:52.018166    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:23:07.498719 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:23:07.498685    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:23:15.692007 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:23:15.691922    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:23:22.603426 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:23:22.603389    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:23:30.791673 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:23:30.791632    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:23:44.398165 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:23:44.398126    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:23:47.210763 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:23:47.210731    2586 scope.go:117] "RemoveContainer" containerID="33d6d67018432ddd4f5c79c4400011dde5e9e2db39ada55b31ad58f503374e84"
Sep 11 12:23:52.993816 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:23:52.993787    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:24:31.622269 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:24:31.622230    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:25:21.588479 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:21.588444    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:25:28.393567 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:28.393534    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:25:35.801362 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:35.801325    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:25:37.797096 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.797059    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"]
Sep 11 12:25:37.797684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.797464    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c586c28c-3d48-4e84-8e10-6148f6e7f84e" containerName="payload-pre-processing"
Sep 11 12:25:37.797684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.797511    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c586c28c-3d48-4e84-8e10-6148f6e7f84e" containerName="payload-pre-processing"
Sep 11 12:25:37.797684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.797533    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795db35f-0c47-4923-9f75-ec435f5ff8a5" containerName="revoke-keys"
Sep 11 12:25:37.797684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.797540    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="795db35f-0c47-4923-9f75-ec435f5ff8a5" containerName="revoke-keys"
Sep 11 12:25:37.797684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.797626    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c586c28c-3d48-4e84-8e10-6148f6e7f84e" containerName="payload-pre-processing"
Sep 11 12:25:37.797684 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.797638    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="795db35f-0c47-4923-9f75-ec435f5ff8a5" containerName="revoke-keys"
Sep 11 12:25:37.800976 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.800955    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.804043 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.804021    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-b958743e-kserve-self-signed-certs\""
Sep 11 12:25:37.809961 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.809940    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"]
Sep 11 12:25:37.832666 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.832644    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:25:37.841828 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.841804    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-dshm\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.841918 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.841839    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-model-cache\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.841918 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.841863    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tls-certs\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.841918 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.841882    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jfzn\" (UniqueName: \"kubernetes.io/projected/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-kube-api-access-5jfzn\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.841918 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.841916    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-home\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.842059 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.841982    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tmp-dir\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942308 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942269    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-home\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942423 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942349    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tmp-dir\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942423 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942402    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-dshm\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942582 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942432    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-model-cache\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942582 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942470    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tls-certs\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942582 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942524    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jfzn\" (UniqueName: \"kubernetes.io/projected/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-kube-api-access-5jfzn\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942733 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942708    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-home\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942801 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942739    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tmp-dir\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.942893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.942873    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-model-cache\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.944945 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.944927    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-dshm\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.945226 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.945208    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tls-certs\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:37.951551 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:37.951533    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jfzn\" (UniqueName: \"kubernetes.io/projected/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-kube-api-access-5jfzn\") pod \"e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") " pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:38.111462 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:38.111386    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:38.384340 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:38.384312    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"]
Sep 11 12:25:39.105473 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:39.105443    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx" event={"ID":"de1762e8-c4f6-4d00-a5ec-3bba99e7f942","Type":"ContainerStarted","Data":"f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8"}
Sep 11 12:25:39.105829 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:39.105492    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx" event={"ID":"de1762e8-c4f6-4d00-a5ec-3bba99e7f942","Type":"ContainerStarted","Data":"c49b968db38ecbcf39cb476d85d4d3119ff26116f4ca7504a9dcebaeb3da8d75"}
Sep 11 12:25:39.123073 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:39.123026    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx" podStartSLOduration=2.123011073 podStartE2EDuration="2.123011073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:25:39.122295404 +0000 UTC m=+2032.832966744" watchObservedRunningTime="2026-09-11 12:25:39.123011073 +0000 UTC m=+2032.833682411"
Sep 11 12:25:40.098228 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:40.098184    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:25:47.235284 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:47.235247    2586 scope.go:117] "RemoveContainer" containerID="8ab3ac319e6408b353ccefdcc8c4a56a492d49a1e4cf36bf9c88e5a74e9dfd98"
Sep 11 12:25:48.112479 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:48.112443    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:48.112664 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:48.112513    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:48.124963 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:48.124939    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:48.157172 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:48.157153    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:25:53.216386 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.216353    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"]
Sep 11 12:25:53.219974 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.219951    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.224935 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.224913    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-b958743e-kserve-self-signed-certs\""
Sep 11 12:25:53.235144 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.235120    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"]
Sep 11 12:25:53.268881 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.268853    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:25:53.369525 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.369459    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9qck\" (UniqueName: \"kubernetes.io/projected/6348041c-1f66-4a72-80e5-c863d360780f-kube-api-access-g9qck\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.369525 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.369521    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-model-cache\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.369673 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.369546    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348041c-1f66-4a72-80e5-c863d360780f-tls-certs\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.369673 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.369573    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-home\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.369673 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.369611    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-dshm\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.369818 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.369722    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-tmp-dir\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.470841 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.470777    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9qck\" (UniqueName: \"kubernetes.io/projected/6348041c-1f66-4a72-80e5-c863d360780f-kube-api-access-g9qck\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.470841 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.470818    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-model-cache\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.470988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.470841    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348041c-1f66-4a72-80e5-c863d360780f-tls-certs\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.470988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.470873    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-home\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.470988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.470915    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-dshm\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.470988 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.470977    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-tmp-dir\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.471224 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.471203    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-model-cache\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.471284 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.471248    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-home\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.471378 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.471360    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-tmp-dir\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.473319 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.473289    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-dshm\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.473635 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.473617    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348041c-1f66-4a72-80e5-c863d360780f-tls-certs\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.480832 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.480797    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9qck\" (UniqueName: \"kubernetes.io/projected/6348041c-1f66-4a72-80e5-c863d360780f-kube-api-access-g9qck\") pod \"e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") " pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.529694 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.529672    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:25:53.800966 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:53.800889    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"]
Sep 11 12:25:53.803157 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:25:53.803135    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6348041c_1f66_4a72_80e5_c863d360780f.slice/crio-1b7f06fd48610d4080aba772bc4e367ee164cca4cd09a84b009fda8a1cef36df WatchSource:0}: Error finding container 1b7f06fd48610d4080aba772bc4e367ee164cca4cd09a84b009fda8a1cef36df: Status 404 returned error can't find the container with id 1b7f06fd48610d4080aba772bc4e367ee164cca4cd09a84b009fda8a1cef36df
Sep 11 12:25:54.176687 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:54.176598    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8" event={"ID":"6348041c-1f66-4a72-80e5-c863d360780f","Type":"ContainerStarted","Data":"4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c"}
Sep 11 12:25:54.176687 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:54.176643    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8" event={"ID":"6348041c-1f66-4a72-80e5-c863d360780f","Type":"ContainerStarted","Data":"1b7f06fd48610d4080aba772bc4e367ee164cca4cd09a84b009fda8a1cef36df"}
Sep 11 12:25:54.196082 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:54.196031    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8" podStartSLOduration=1.196016539 podStartE2EDuration="1.196016539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:25:54.194100038 +0000 UTC m=+2047.904771381" watchObservedRunningTime="2026-09-11 12:25:54.196016539 +0000 UTC m=+2047.906687876"
Sep 11 12:25:55.916540 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:25:55.916505    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:26:03.530362 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:03.530326    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:26:03.530974 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:03.530378    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:26:03.542395 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:03.542371    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:26:04.238074 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:04.238038    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:26:05.972428 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:05.972395    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:26:26.288387 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:26.288302    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"]
Sep 11 12:26:26.309607 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:26.309579    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:26:27.420911 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:27.420878    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"]
Sep 11 12:26:27.447460 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:27.447425    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:26:29.210861 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:29.210826    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:26:36.992995 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:36.992910    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:26:41.288457 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.288410    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:26:41.308518 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.308450    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx" podUID="de1762e8-c4f6-4d00-a5ec-3bba99e7f942" containerName="main" containerID="cri-o://f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8" gracePeriod=45
Sep 11 12:26:41.561093 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.561062    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:26:41.657688 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.657664    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tls-certs\") pod \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") "
Sep 11 12:26:41.657824 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.657721    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-dshm\") pod \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") "
Sep 11 12:26:41.657824 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.657749    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jfzn\" (UniqueName: \"kubernetes.io/projected/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-kube-api-access-5jfzn\") pod \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") "
Sep 11 12:26:41.657824 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.657770    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tmp-dir\") pod \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") "
Sep 11 12:26:41.657824 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.657791    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-model-cache\") pod \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") "
Sep 11 12:26:41.658026 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.657894    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-home\") pod \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\" (UID: \"de1762e8-c4f6-4d00-a5ec-3bba99e7f942\") "
Sep 11 12:26:41.658144 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.658085    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de1762e8-c4f6-4d00-a5ec-3bba99e7f942" (UID: "de1762e8-c4f6-4d00-a5ec-3bba99e7f942"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:41.658230 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.658150    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-model-cache" (OuterVolumeSpecName: "model-cache") pod "de1762e8-c4f6-4d00-a5ec-3bba99e7f942" (UID: "de1762e8-c4f6-4d00-a5ec-3bba99e7f942"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:41.658230 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.658195    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-home" (OuterVolumeSpecName: "home") pod "de1762e8-c4f6-4d00-a5ec-3bba99e7f942" (UID: "de1762e8-c4f6-4d00-a5ec-3bba99e7f942"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:41.658695 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.658675    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:41.658765 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.658702    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:41.658765 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.658718    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:41.660008 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.659983    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-dshm" (OuterVolumeSpecName: "dshm") pod "de1762e8-c4f6-4d00-a5ec-3bba99e7f942" (UID: "de1762e8-c4f6-4d00-a5ec-3bba99e7f942"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:41.660123 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.659987    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de1762e8-c4f6-4d00-a5ec-3bba99e7f942" (UID: "de1762e8-c4f6-4d00-a5ec-3bba99e7f942"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:26:41.660356 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.660336    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-kube-api-access-5jfzn" (OuterVolumeSpecName: "kube-api-access-5jfzn") pod "de1762e8-c4f6-4d00-a5ec-3bba99e7f942" (UID: "de1762e8-c4f6-4d00-a5ec-3bba99e7f942"). InnerVolumeSpecName "kube-api-access-5jfzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:26:41.760158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.760126    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:41.760158 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.760159    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:41.760342 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:41.760174    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jfzn\" (UniqueName: \"kubernetes.io/projected/de1762e8-c4f6-4d00-a5ec-3bba99e7f942-kube-api-access-5jfzn\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:42.418429 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.418393    2586 generic.go:356] "Generic (PLEG): container finished" podID="de1762e8-c4f6-4d00-a5ec-3bba99e7f942" containerID="f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8" exitCode=0
Sep 11 12:26:42.418903 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.418452    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"
Sep 11 12:26:42.418903 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.418467    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx" event={"ID":"de1762e8-c4f6-4d00-a5ec-3bba99e7f942","Type":"ContainerDied","Data":"f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8"}
Sep 11 12:26:42.418903 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.418524    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx" event={"ID":"de1762e8-c4f6-4d00-a5ec-3bba99e7f942","Type":"ContainerDied","Data":"c49b968db38ecbcf39cb476d85d4d3119ff26116f4ca7504a9dcebaeb3da8d75"}
Sep 11 12:26:42.418903 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.418542    2586 scope.go:117] "RemoveContainer" containerID="f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8"
Sep 11 12:26:42.427943 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.427927    2586 scope.go:117] "RemoveContainer" containerID="f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8"
Sep 11 12:26:42.428257 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:26:42.428236    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8\": container with ID starting with f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8 not found: ID does not exist" containerID="f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8"
Sep 11 12:26:42.428307 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.428265    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8"} err="failed to get container status \"f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8\": rpc error: code = NotFound desc = could not find container \"f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8\": container with ID starting with f0420dcae923777f88f853846b58d435ed1aaf93a26ed67d4c0cdab388af01b8 not found: ID does not exist"
Sep 11 12:26:42.440245 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.440224    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"]
Sep 11 12:26:42.440587 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.440560    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8" podUID="6348041c-1f66-4a72-80e5-c863d360780f" containerName="main" containerID="cri-o://4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c" gracePeriod=45
Sep 11 12:26:42.444133 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.444114    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-b958743e-kserve-756b4fdc59-knzdx"]
Sep 11 12:26:42.699398 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.699368    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:26:42.767396 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767374    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-dshm\") pod \"6348041c-1f66-4a72-80e5-c863d360780f\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") "
Sep 11 12:26:42.767556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767420    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-home\") pod \"6348041c-1f66-4a72-80e5-c863d360780f\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") "
Sep 11 12:26:42.767556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767510    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9qck\" (UniqueName: \"kubernetes.io/projected/6348041c-1f66-4a72-80e5-c863d360780f-kube-api-access-g9qck\") pod \"6348041c-1f66-4a72-80e5-c863d360780f\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") "
Sep 11 12:26:42.767556 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767544    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348041c-1f66-4a72-80e5-c863d360780f-tls-certs\") pod \"6348041c-1f66-4a72-80e5-c863d360780f\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") "
Sep 11 12:26:42.767726 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767574    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-tmp-dir\") pod \"6348041c-1f66-4a72-80e5-c863d360780f\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") "
Sep 11 12:26:42.767726 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767598    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-model-cache\") pod \"6348041c-1f66-4a72-80e5-c863d360780f\" (UID: \"6348041c-1f66-4a72-80e5-c863d360780f\") "
Sep 11 12:26:42.767838 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767720    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-home" (OuterVolumeSpecName: "home") pod "6348041c-1f66-4a72-80e5-c863d360780f" (UID: "6348041c-1f66-4a72-80e5-c863d360780f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:42.767903 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767828    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6348041c-1f66-4a72-80e5-c863d360780f" (UID: "6348041c-1f66-4a72-80e5-c863d360780f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:42.767903 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767890    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:42.767987 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.767906    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6348041c-1f66-4a72-80e5-c863d360780f" (UID: "6348041c-1f66-4a72-80e5-c863d360780f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:42.769647 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.769622    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-dshm" (OuterVolumeSpecName: "dshm") pod "6348041c-1f66-4a72-80e5-c863d360780f" (UID: "6348041c-1f66-4a72-80e5-c863d360780f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:26:42.769757 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.769685    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6348041c-1f66-4a72-80e5-c863d360780f-kube-api-access-g9qck" (OuterVolumeSpecName: "kube-api-access-g9qck") pod "6348041c-1f66-4a72-80e5-c863d360780f" (UID: "6348041c-1f66-4a72-80e5-c863d360780f"). InnerVolumeSpecName "kube-api-access-g9qck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:26:42.769847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.769817    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6348041c-1f66-4a72-80e5-c863d360780f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6348041c-1f66-4a72-80e5-c863d360780f" (UID: "6348041c-1f66-4a72-80e5-c863d360780f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:26:42.824041 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.824008    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de1762e8-c4f6-4d00-a5ec-3bba99e7f942" path="/var/lib/kubelet/pods/de1762e8-c4f6-4d00-a5ec-3bba99e7f942/volumes"
Sep 11 12:26:42.868379 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.868356    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9qck\" (UniqueName: \"kubernetes.io/projected/6348041c-1f66-4a72-80e5-c863d360780f-kube-api-access-g9qck\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:42.868379 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.868377    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348041c-1f66-4a72-80e5-c863d360780f-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:42.868542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.868385    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:42.868542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.868394    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:42.868542 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:42.868402    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348041c-1f66-4a72-80e5-c863d360780f-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Sep 11 12:26:43.425501 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.425459    2586 generic.go:356] "Generic (PLEG): container finished" podID="6348041c-1f66-4a72-80e5-c863d360780f" containerID="4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c" exitCode=0
Sep 11 12:26:43.425874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.425517    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"
Sep 11 12:26:43.425874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.425523    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8" event={"ID":"6348041c-1f66-4a72-80e5-c863d360780f","Type":"ContainerDied","Data":"4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c"}
Sep 11 12:26:43.425874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.425567    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8" event={"ID":"6348041c-1f66-4a72-80e5-c863d360780f","Type":"ContainerDied","Data":"1b7f06fd48610d4080aba772bc4e367ee164cca4cd09a84b009fda8a1cef36df"}
Sep 11 12:26:43.425874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.425583    2586 scope.go:117] "RemoveContainer" containerID="4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c"
Sep 11 12:26:43.433958 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.433936    2586 scope.go:117] "RemoveContainer" containerID="4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c"
Sep 11 12:26:43.434221 ip-10-0-130-2 kubenswrapper[2586]: E0911 12:26:43.434202    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c\": container with ID starting with 4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c not found: ID does not exist" containerID="4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c"
Sep 11 12:26:43.434279 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.434233    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c"} err="failed to get container status \"4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c\": rpc error: code = NotFound desc = could not find container \"4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c\": container with ID starting with 4e094c5268400e7ccedea813cc485a7a66819a2e899784b162fd11176f545b1c not found: ID does not exist"
Sep 11 12:26:43.443164 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.443143    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"]
Sep 11 12:26:43.447341 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:43.447317    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-b958743e-kserve-78566d7559-9hrm8"]
Sep 11 12:26:44.502287 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:44.502255    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:26:44.824045 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:44.824006    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6348041c-1f66-4a72-80e5-c863d360780f" path="/var/lib/kubelet/pods/6348041c-1f66-4a72-80e5-c863d360780f/volumes"
Sep 11 12:26:46.931845 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:46.931816    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:26:46.939744 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:46.939723    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-92cnc_cd20463d-4e13-4c7d-8f18-9bc131b4f41f/ovn-acl-logging/0.log"
Sep 11 12:26:47.253134 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:47.253039    2586 scope.go:117] "RemoveContainer" containerID="5e0e324c8eb49ff4c3ca6e81247c51887f67501401a7b80bcb7e33a50983accb"
Sep 11 12:26:47.260938 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:47.260919    2586 scope.go:117] "RemoveContainer" containerID="75c8b2d758439a591eaced0f73b7713a4f98f9c72eb99b961b586f860b569930"
Sep 11 12:26:52.302087 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:52.302050    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:26:52.323797 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:52.323768    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:26:59.894944 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:26:59.894909    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:27:08.222613 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:08.222578    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:27:15.502025 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:15.501989    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xxwdg"]
Sep 11 12:27:18.994823 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:18.994789    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bb5846b4f-rsb2m_3cab8006-af4c-43a4-9f71-d015f2e4b60e/authorino/0.log"
Sep 11 12:27:25.791708 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:25.791678    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb_c8f152b3-e789-4590-ab2a-d08d0423e646/copy-manifests/0.log"
Sep 11 12:27:25.803251 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:25.803223    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb_c8f152b3-e789-4590-ab2a-d08d0423e646/manager/0.log"
Sep 11 12:27:27.318408 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:27.318372    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bb5846b4f-rsb2m_3cab8006-af4c-43a4-9f71-d015f2e4b60e/authorino/0.log"
Sep 11 12:27:27.665547 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:27.665457    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-x89n9_7835421c-171e-43fe-ac86-e9d57be651c2/kuadrant-console-plugin/0.log"
Sep 11 12:27:27.928985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:27.928909    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-xxwdg_decef843-bc3d-4301-b784-c41b671ba53c/limitador/0.log"
Sep 11 12:27:28.842071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:28.842044    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8478ddd598-tzln4_b70f06c0-3711-470f-bd0a-deaf9877361f/kube-auth-proxy/0.log"
Sep 11 12:27:29.131973 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:29.131900    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vtc56_a27fa3b8-75aa-4764-8995-f5eaf0240260/payload-pre-processing/0.log"
Sep 11 12:27:29.380445 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:29.380422    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f585bfd8b-vmzws_8a934645-42a5-466a-928f-acfe9ba67c9a/router/0.log"
Sep 11 12:27:29.989010 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:29.988981    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-zrtx9_9da20359-b125-441f-9590-8619b726330f/main/0.log"
Sep 11 12:27:30.475498 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:30.475455    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-4ns4w_8cdd77ab-5abf-4d1a-8491-3d8d02f4c53b/main/0.log"
Sep 11 12:27:36.217327 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.217290    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rhtnl/must-gather-kbk88"]
Sep 11 12:27:36.217735 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.217679    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de1762e8-c4f6-4d00-a5ec-3bba99e7f942" containerName="main"
Sep 11 12:27:36.217735 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.217718    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="de1762e8-c4f6-4d00-a5ec-3bba99e7f942" containerName="main"
Sep 11 12:27:36.217735 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.217730    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6348041c-1f66-4a72-80e5-c863d360780f" containerName="main"
Sep 11 12:27:36.217735 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.217736    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6348041c-1f66-4a72-80e5-c863d360780f" containerName="main"
Sep 11 12:27:36.217893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.217787    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="de1762e8-c4f6-4d00-a5ec-3bba99e7f942" containerName="main"
Sep 11 12:27:36.217893 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.217796    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6348041c-1f66-4a72-80e5-c863d360780f" containerName="main"
Sep 11 12:27:36.220773 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.220753    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.223758 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.223737    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rhtnl\"/\"kube-root-ca.crt\""
Sep 11 12:27:36.223975 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.223960    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rhtnl\"/\"default-dockercfg-blx5s\""
Sep 11 12:27:36.224057 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.224028    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rhtnl\"/\"openshift-service-ca.crt\""
Sep 11 12:27:36.231985 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.231963    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rhtnl/must-gather-kbk88"]
Sep 11 12:27:36.236535 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.236516    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:27:36.304152 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.304128    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22dpz\" (UniqueName: \"kubernetes.io/projected/71182647-013d-4af3-b1c5-4de2203a4c3d-kube-api-access-22dpz\") pod \"must-gather-kbk88\" (UID: \"71182647-013d-4af3-b1c5-4de2203a4c3d\") " pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.304252 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.304157    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/71182647-013d-4af3-b1c5-4de2203a4c3d-must-gather-output\") pod \"must-gather-kbk88\" (UID: \"71182647-013d-4af3-b1c5-4de2203a4c3d\") " pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.309010 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.308991    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log" limit=1024
Sep 11 12:27:36.404821 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.404789    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22dpz\" (UniqueName: \"kubernetes.io/projected/71182647-013d-4af3-b1c5-4de2203a4c3d-kube-api-access-22dpz\") pod \"must-gather-kbk88\" (UID: \"71182647-013d-4af3-b1c5-4de2203a4c3d\") " pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.404939 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.404831    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/71182647-013d-4af3-b1c5-4de2203a4c3d-must-gather-output\") pod \"must-gather-kbk88\" (UID: \"71182647-013d-4af3-b1c5-4de2203a4c3d\") " pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.405145 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.405126    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/71182647-013d-4af3-b1c5-4de2203a4c3d-must-gather-output\") pod \"must-gather-kbk88\" (UID: \"71182647-013d-4af3-b1c5-4de2203a4c3d\") " pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.413139 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.413120    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22dpz\" (UniqueName: \"kubernetes.io/projected/71182647-013d-4af3-b1c5-4de2203a4c3d-kube-api-access-22dpz\") pod \"must-gather-kbk88\" (UID: \"71182647-013d-4af3-b1c5-4de2203a4c3d\") " pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.530028 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.529970    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rhtnl/must-gather-kbk88"
Sep 11 12:27:36.801764 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.799049    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rhtnl/must-gather-kbk88"]
Sep 11 12:27:36.801764 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:27:36.801069    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71182647_013d_4af3_b1c5_4de2203a4c3d.slice/crio-b263a6899e109904183ca23a1df94190ee26f2d1dd263b8c8896d9d6cd7a0602 WatchSource:0}: Error finding container b263a6899e109904183ca23a1df94190ee26f2d1dd263b8c8896d9d6cd7a0602: Status 404 returned error can't find the container with id b263a6899e109904183ca23a1df94190ee26f2d1dd263b8c8896d9d6cd7a0602
Sep 11 12:27:36.802979 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:36.802958    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:27:37.684546 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:37.684474    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rhtnl/must-gather-kbk88" event={"ID":"71182647-013d-4af3-b1c5-4de2203a4c3d","Type":"ContainerStarted","Data":"0300b86a24724fdca79bb4fc631c2acefc76675f72fc40dad9c3a634285032d9"}
Sep 11 12:27:37.684546 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:37.684548    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rhtnl/must-gather-kbk88" event={"ID":"71182647-013d-4af3-b1c5-4de2203a4c3d","Type":"ContainerStarted","Data":"b263a6899e109904183ca23a1df94190ee26f2d1dd263b8c8896d9d6cd7a0602"}
Sep 11 12:27:38.696409 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:38.696326    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rhtnl/must-gather-kbk88" event={"ID":"71182647-013d-4af3-b1c5-4de2203a4c3d","Type":"ContainerStarted","Data":"ab32af41fc66042de9104876e54b50c742ab1f26921a6f392d95329bc1b14514"}
Sep 11 12:27:38.713091 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:38.713037    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rhtnl/must-gather-kbk88" podStartSLOduration=2.003959936 podStartE2EDuration="2.71302047s" totalImagesPullingTime="709.060534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:27:38.712514166 +0000 UTC m=+2152.423185504" watchObservedRunningTime="2026-09-11 12:27:38.71302047 +0000 UTC m=+2152.423691808"
Sep 11 12:27:39.075373 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:39.075335    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xq77w_49f88faf-4689-4ca0-a18a-4cb7d895eea8/global-pull-secret-syncer/0.log"
Sep 11 12:27:39.160874 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:39.160834    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-22ffn_34426957-e64e-412e-b12f-6c27280a60e8/konnectivity-agent/0.log"
Sep 11 12:27:39.266156 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:39.266122    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-2.ec2.internal_d43279f9f2fc94f3afdb8b4c02cff3d7/haproxy/0.log"
Sep 11 12:27:43.594048 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:43.593025    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bb5846b4f-rsb2m_3cab8006-af4c-43a4-9f71-d015f2e4b60e/authorino/0.log"
Sep 11 12:27:43.762248 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:43.757263    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-x89n9_7835421c-171e-43fe-ac86-e9d57be651c2/kuadrant-console-plugin/0.log"
Sep 11 12:27:43.839929 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:43.839814    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-xxwdg_decef843-bc3d-4301-b784-c41b671ba53c/limitador/0.log"
Sep 11 12:27:45.697803 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:45.697746    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-zcdrs_ac6fe090-abfb-42d6-995b-0ab2da0c0969/kube-state-metrics/0.log"
Sep 11 12:27:45.724439 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:45.724397    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-zcdrs_ac6fe090-abfb-42d6-995b-0ab2da0c0969/kube-rbac-proxy-main/0.log"
Sep 11 12:27:45.748725 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:45.748611    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-zcdrs_ac6fe090-abfb-42d6-995b-0ab2da0c0969/kube-rbac-proxy-self/0.log"
Sep 11 12:27:45.800680 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:45.800629    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7c6789cfb-g5mtr_d5919dee-c964-496b-b29e-cbe63ba6fdfd/metrics-server/0.log"
Sep 11 12:27:45.832513 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:45.832468    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-dbrcm_c293a891-acda-430a-b509-a0c05d9ac9e8/monitoring-plugin/0.log"
Sep 11 12:27:46.007782 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.007674    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxrzn_71251815-9411-41cb-bb13-30c51f158ebe/node-exporter/0.log"
Sep 11 12:27:46.034135 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.034105    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxrzn_71251815-9411-41cb-bb13-30c51f158ebe/kube-rbac-proxy/0.log"
Sep 11 12:27:46.059443 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.059410    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxrzn_71251815-9411-41cb-bb13-30c51f158ebe/init-textfile/0.log"
Sep 11 12:27:46.176703 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.176668    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_83d9eb10-477e-4483-bb40-1412d7882842/prometheus/0.log"
Sep 11 12:27:46.198653 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.198619    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_83d9eb10-477e-4483-bb40-1412d7882842/config-reloader/0.log"
Sep 11 12:27:46.223824 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.223793    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_83d9eb10-477e-4483-bb40-1412d7882842/thanos-sidecar/0.log"
Sep 11 12:27:46.256558 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.256527    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_83d9eb10-477e-4483-bb40-1412d7882842/kube-rbac-proxy-web/0.log"
Sep 11 12:27:46.278543 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.278450    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_83d9eb10-477e-4483-bb40-1412d7882842/kube-rbac-proxy/0.log"
Sep 11 12:27:46.301651 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.301627    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_83d9eb10-477e-4483-bb40-1412d7882842/kube-rbac-proxy-thanos/0.log"
Sep 11 12:27:46.323913 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:46.323883    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_83d9eb10-477e-4483-bb40-1412d7882842/init-config-reloader/0.log"
Sep 11 12:27:47.289834 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:47.289807    2586 scope.go:117] "RemoveContainer" containerID="e28a11c87e8a223ae1dae2664d98181e3bfb7d06e6d5b914d060c6bc0b5998c6"
Sep 11 12:27:49.527244 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.527215    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-54c6bd499-cfw7b_cc41508a-27b8-455f-96d2-49f922e972be/console/0.log"
Sep 11 12:27:49.563700 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.563670    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-6dwvk_fb40254a-26c2-495e-bc2c-48f1d22ed985/download-server/0.log"
Sep 11 12:27:49.760650 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.760613    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"]
Sep 11 12:27:49.768820 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.768783    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.777329 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.777246    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"]
Sep 11 12:27:49.861347 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.861299    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-podres\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.861553 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.861429    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgsm7\" (UniqueName: \"kubernetes.io/projected/18d786a8-780c-4532-abf0-8074deee0082-kube-api-access-xgsm7\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.861553 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.861538    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-sys\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.861667 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.861596    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-lib-modules\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.861667 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.861629    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-proc\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.962791 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.962745    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgsm7\" (UniqueName: \"kubernetes.io/projected/18d786a8-780c-4532-abf0-8074deee0082-kube-api-access-xgsm7\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.963913 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.963885    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-sys\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.964348 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.964330    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-sys\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.964631 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.964614    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-lib-modules\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.964982 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.964871    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-lib-modules\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.965165 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.965143    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-proc\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.965378 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.965298    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-proc\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.965616 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.965437    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-podres\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.965712 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.965544    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18d786a8-780c-4532-abf0-8074deee0082-podres\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:49.972562 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:49.972543    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgsm7\" (UniqueName: \"kubernetes.io/projected/18d786a8-780c-4532-abf0-8074deee0082-kube-api-access-xgsm7\") pod \"perf-node-gather-daemonset-m72wh\" (UID: \"18d786a8-780c-4532-abf0-8074deee0082\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:50.082605 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:50.082569    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:50.397291 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:50.397194    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"]
Sep 11 12:27:50.410185 ip-10-0-130-2 kubenswrapper[2586]: W0911 12:27:50.410136    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18d786a8_780c_4532_abf0_8074deee0082.slice/crio-5cc75bf6d51494108260b7babb8efad096535209900d45ed1748c4fb8e1265df WatchSource:0}: Error finding container 5cc75bf6d51494108260b7babb8efad096535209900d45ed1748c4fb8e1265df: Status 404 returned error can't find the container with id 5cc75bf6d51494108260b7babb8efad096535209900d45ed1748c4fb8e1265df
Sep 11 12:27:50.808419 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:50.808368    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh" event={"ID":"18d786a8-780c-4532-abf0-8074deee0082","Type":"ContainerStarted","Data":"814d0615e0242374c4b528ed120361b49c82c456becde31d73f0618370ef8c96"}
Sep 11 12:27:50.808915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:50.808437    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh" event={"ID":"18d786a8-780c-4532-abf0-8074deee0082","Type":"ContainerStarted","Data":"5cc75bf6d51494108260b7babb8efad096535209900d45ed1748c4fb8e1265df"}
Sep 11 12:27:50.808915 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:50.808699    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:50.828243 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:50.828162    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh" podStartSLOduration=1.828139628 podStartE2EDuration="1.828139628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:27:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:27:50.826719883 +0000 UTC m=+2164.537391225" watchObservedRunningTime="2026-09-11 12:27:50.828139628 +0000 UTC m=+2164.538810966"
Sep 11 12:27:51.631325 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:51.631289    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ss9wb_2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a/dns/0.log"
Sep 11 12:27:51.653567 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:51.653536    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ss9wb_2dd6e280-bc22-4dc4-bf7c-47a8adb34b4a/kube-rbac-proxy/0.log"
Sep 11 12:27:51.702431 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:51.702392    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzkmp_21124859-d604-4df1-b825-d6dbfb5ecace/dns-node-resolver/0.log"
Sep 11 12:27:52.358267 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:52.358243    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qqt5h_868bf557-a754-4bc4-92bb-54bf847ab8f7/node-ca/0.log"
Sep 11 12:27:53.757054 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:53.757022    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8478ddd598-tzln4_b70f06c0-3711-470f-bd0a-deaf9877361f/kube-auth-proxy/0.log"
Sep 11 12:27:53.845808 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:53.845779    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vtc56_a27fa3b8-75aa-4764-8995-f5eaf0240260/payload-pre-processing/0.log"
Sep 11 12:27:53.921136 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:53.921089    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f585bfd8b-vmzws_8a934645-42a5-466a-928f-acfe9ba67c9a/router/0.log"
Sep 11 12:27:54.845101 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:54.845071    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l68qh_71cca530-f27e-4d2a-8a03-e6a7d99064a5/serve-healthcheck-canary/0.log"
Sep 11 12:27:55.470602 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:55.470551    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pxlwr_22381901-1722-4032-888a-6ac1a3868149/kube-rbac-proxy/0.log"
Sep 11 12:27:55.498635 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:55.498611    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pxlwr_22381901-1722-4032-888a-6ac1a3868149/exporter/0.log"
Sep 11 12:27:55.523532 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:55.523506    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pxlwr_22381901-1722-4032-888a-6ac1a3868149/extractor/0.log"
Sep 11 12:27:56.826623 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:56.826595    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-m72wh"
Sep 11 12:27:58.412183 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:58.412144    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb_c8f152b3-e789-4590-ab2a-d08d0423e646/manager/0.log"
Sep 11 12:27:58.432749 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:58.432722    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-sn2wb_c8f152b3-e789-4590-ab2a-d08d0423e646/copy-manifests/0.log"
Sep 11 12:27:59.884377 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:27:59.884337    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56c7cddbd4-kmkxj_1388bfb1-b0b0-41e6-a151-4a7434f0437b/manager/0.log"
Sep 11 12:28:00.663020 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:00.662989    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8478ddd598-tzln4_b70f06c0-3711-470f-bd0a-deaf9877361f/kube-auth-proxy/0.log"
Sep 11 12:28:00.726328 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:00.726294    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vtc56_a27fa3b8-75aa-4764-8995-f5eaf0240260/payload-pre-processing/0.log"
Sep 11 12:28:00.788074 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:00.788042    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f585bfd8b-vmzws_8a934645-42a5-466a-928f-acfe9ba67c9a/router/0.log"
Sep 11 12:28:02.804186 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:02.804152    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8478ddd598-tzln4_b70f06c0-3711-470f-bd0a-deaf9877361f/kube-auth-proxy/0.log"
Sep 11 12:28:02.888051 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:02.888021    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vtc56_a27fa3b8-75aa-4764-8995-f5eaf0240260/payload-pre-processing/0.log"
Sep 11 12:28:02.947071 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:02.947040    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f585bfd8b-vmzws_8a934645-42a5-466a-928f-acfe9ba67c9a/router/0.log"
Sep 11 12:28:06.303294 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.303267    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fxsgk_2b40db0d-a049-4c64-ad35-463fc15f3bbc/kube-multus-additional-cni-plugins/0.log"
Sep 11 12:28:06.324645 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.324622    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fxsgk_2b40db0d-a049-4c64-ad35-463fc15f3bbc/egress-router-binary-copy/0.log"
Sep 11 12:28:06.347797 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.347779    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fxsgk_2b40db0d-a049-4c64-ad35-463fc15f3bbc/cni-plugins/0.log"
Sep 11 12:28:06.367933 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.367910    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fxsgk_2b40db0d-a049-4c64-ad35-463fc15f3bbc/bond-cni-plugin/0.log"
Sep 11 12:28:06.388670 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.388656    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fxsgk_2b40db0d-a049-4c64-ad35-463fc15f3bbc/routeoverride-cni/0.log"
Sep 11 12:28:06.408574 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.408547    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fxsgk_2b40db0d-a049-4c64-ad35-463fc15f3bbc/whereabouts-cni-bincopy/0.log"
Sep 11 12:28:06.428746 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.428728    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fxsgk_2b40db0d-a049-4c64-ad35-463fc15f3bbc/whereabouts-cni/0.log"
Sep 11 12:28:06.639847 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.639768    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-msxfk_18a3f084-1e1f-42d4-bdad-fe521b4cd471/kube-multus/0.log"
Sep 11 12:28:06.827886 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.827803    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v66kj_dfb47d93-fe8c-42eb-a7f7-0897d42c72ff/network-metrics-daemon/0.log"
Sep 11 12:28:06.844689 ip-10-0-130-2 kubenswrapper[2586]: I0911 12:28:06.844658    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v66kj_dfb47d93-fe8c-42eb-a7f7-0897d42c72ff/kube-rbac-proxy/0.log"