2026-08-05T17:58:00.864672611Z [disk usage checker] Started
2026-08-05T17:58:00.877219814Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-08-05T17:58:01.072415895Z Gathering data for ns/openshift-cluster-version...
2026-08-05T17:58:01.214895075Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-05T17:58:01.304130985Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-05T17:58:01.386574704Z Gathering data for ns/default...
2026-08-05T17:58:01.672732215Z Gathering data for ns/openshift...
2026-08-05T17:58:02.120782689Z Gathering data for ns/kube-system...
2026-08-05T17:58:02.896467228Z Gathering data for ns/openshift-etcd...
2026-08-05T17:58:03.223808301Z Wrote inspect data to must-gather.
2026-08-05T17:58:03.223938951Z error: inspection completed with the errors occurred while gathering data:
2026-08-05T17:58:03.223938951Z     namespaces "assisted-installer" not found
2026-08-05T17:58:03.595575808Z Waiting on subprocesses to finish execution.
2026-08-05T17:58:03.603160879Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-08-05T17:58:03.606206162Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-08-05T17:58:03.612571000Z INFO: Gathering machine config daemon's old logs from all nodes
2026-08-05T17:58:03.615133403Z INFO: Collecting host service logs for crio
2026-08-05T17:58:03.615433245Z INFO: Collecting host service logs for kubelet
2026-08-05T17:58:03.615830799Z INFO: Collecting host service logs for rpm-ostreed
2026-08-05T17:58:03.616193960Z INFO: Collecting host service logs for ostree-finalize-staged
2026-08-05T17:58:03.616503195Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-08-05T17:58:03.616819985Z INFO: Collecting host service logs for machine-config-daemon-host
2026-08-05T17:58:03.617157788Z INFO: Collecting host service logs for NetworkManager
2026-08-05T17:58:03.617450704Z INFO: Collecting host service logs for openvswitch
2026-08-05T17:58:03.617747357Z INFO: Collecting host service logs for ovs-configuration
2026-08-05T17:58:03.618080981Z INFO: Collecting host service logs for ovsdb-server
2026-08-05T17:58:03.618346605Z INFO: Collecting host service logs for ovs-vswitchd
2026-08-05T17:58:03.618648603Z INFO: Waiting for worker host service log collection to complete ...
2026-08-05T17:58:03.619621675Z Executing Istio gather script
2026-08-05T17:58:03.629103873Z INFO: Gathering HAProxy config files
2026-08-05T17:58:03.665558543Z INFO: Waiting for node performance related collection to complete ...
2026-08-05T17:58:04.117037249Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-08-05T17:58:04.323333433Z No resources found in openshift-etcd namespace.
2026-08-05T17:58:04.453001833Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-08-05T17:58:04.453043218Z INFO: on-disk MachineConfig config collection complete.
2026-08-05T17:58:04.619050131Z INFO: Collecting Insights Archives from insights-operator-77696685b-dj6cq
2026-08-05T17:58:04.619115005Z insights-runtime-extractor-cbrff
2026-08-05T17:58:04.619136413Z insights-runtime-extractor-ghgbw
2026-08-05T17:58:04.619156747Z insights-runtime-extractor-h5kng
2026-08-05T17:58:04.718299563Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-08-05T17:58:04.780465390Z INFO: "metallb-operator" not detected. Skipping.
2026-08-05T17:58:04.985663307Z INFO: Found 1 replicas - prometheus-k8s-0
2026-08-05T17:58:05.043649719Z No resources found
2026-08-05T17:58:05.063068488Z Wrote inspect data to must-gather.
2026-08-05T17:58:05.098411293Z ERROR: No running kube-apiserver pods found
2026-08-05T17:58:05.118774901Z INFO: "sriov-network-operator" not detected. Skipping.
2026-08-05T17:58:05.119909200Z INFO: Worker host service log collection to complete.
2026-08-05T17:58:05.180743941Z INFO: Waiting for HAProxy config collection to complete ...
2026-08-05T17:58:05.220041804Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-08-05T17:58:05.220041804Z See 'oc rsync -h' for help and examples
2026-08-05T17:58:05.231930009Z error: the server doesn't have a resource type "performanceprofile"
2026-08-05T17:58:05.468362757Z Wrote inspect data to must-gather.
2026-08-05T17:58:05.521785885Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-08-05T17:58:05.621712363Z Gathering data for ns/openshift-monitoring...
2026-08-05T17:58:05.720264135Z tar: Removing leading `/' from member names
2026-08-05T17:58:05.725502750Z INFO: HAProxy config collection complete.
2026-08-05T17:58:05.744820853Z Wrote inspect data to must-gather.
2026-08-05T17:58:05.779110427Z INFO: 'previous-logs' folder not found on ip-10-0-135-56.ec2.internal, skipping...
2026-08-05T17:58:05.924999752Z Wrote inspect data to must-gather.
2026-08-05T17:58:05.983270876Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-08-05T17:58:06.005993295Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-05T17:58:06.176287227Z Wrote inspect data to must-gather.
2026-08-05T17:58:06.227695346Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-08-05T17:58:06.229952528Z No resources found
2026-08-05T17:58:06.341126423Z error: the server doesn't have a resource type "machineconfigs"
2026-08-05T17:58:06.362570874Z Wrote inspect data to must-gather.
2026-08-05T17:58:06.464280426Z Wrote inspect data to must-gather.
2026-08-05T17:58:06.520941539Z INFO: Getting rules from prometheus-k8s-0
2026-08-05T17:58:06.544327755Z error: the server doesn't have a resource type "machineconfigpools"
2026-08-05T17:58:06.557683841Z INFO: 'previous-logs' folder not found on ip-10-0-139-158.ec2.internal, skipping...
2026-08-05T17:58:06.722813904Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-08-05T17:58:06.766299101Z Wrote inspect data to must-gather.
2026-08-05T17:58:06.816490479Z Wrote inspect data to must-gather.
2026-08-05T17:58:06.837794980Z INFO: 'previous-logs' folder not found on ip-10-0-143-121.ec2.internal, skipping...
2026-08-05T17:58:06.838525248Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-08-05T17:58:06.838593880Z INFO: Machine Config Daemon termination log collection complete.
2026-08-05T17:58:06.996046275Z error: the server doesn't have a resource type "kubeletconfigs"
2026-08-05T17:58:07.000381601Z INFO: Getting status/config from prometheus-k8s-0
2026-08-05T17:58:07.044751203Z Wrote inspect data to must-gather.
2026-08-05T17:58:07.223121474Z Wrote inspect data to must-gather.
2026-08-05T17:58:07.311161387Z Wrote inspect data to must-gather.
2026-08-05T17:58:07.409662054Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-05T17:58:07.428284787Z INFO: Getting status/flags from prometheus-k8s-0
2026-08-05T17:58:07.470647869Z Wrote inspect data to must-gather.
2026-08-05T17:58:07.613095668Z INFO: INTERCONNECT MODE
2026-08-05T17:58:07.613134383Z INFO: Gathering ovn-kubernetes DBs
2026-08-05T17:58:07.634407310Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-08-05T17:58:07.672608921Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-08-05T17:58:07.699243239Z Wrote inspect data to must-gather.
2026-08-05T17:58:07.795385730Z INFO: Gathering OVN_Northbound from ovnkube-node-9hn4t...
2026-08-05T17:58:07.795657410Z INFO: Gathering OVN_Northbound from ovnkube-node-pjgjr...
2026-08-05T17:58:07.795914765Z INFO: Gathering OVN_Northbound from ovnkube-node-r69sp...
2026-08-05T17:58:07.796207590Z INFO: Gathering OVN_Southbound from ovnkube-node-9hn4t...
2026-08-05T17:58:07.796515755Z INFO: Gathering OVN_Southbound from ovnkube-node-pjgjr...
2026-08-05T17:58:07.796802200Z INFO: Gathering OVN_Southbound from ovnkube-node-r69sp...
2026-08-05T17:58:07.853625341Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:07.958560053Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-08-05T17:58:08.044193518Z W0805 17:58:08.044149    1340 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-08-05T17:58:08.056886027Z Wrote inspect data to must-gather.
2026-08-05T17:58:08.089340946Z tar: Removing leading `/' from member names
2026-08-05T17:58:08.127723253Z tar: Removing leading `/' from member names
2026-08-05T17:58:08.173202917Z tar: Removing leading `/' from member names
2026-08-05T17:58:08.185635804Z tar: Removing leading `/' from member names
2026-08-05T17:58:08.204728923Z tar: Removing leading `/' from member names
2026-08-05T17:58:08.208588255Z tar: Removing leading `/' from member names
2026-08-05T17:58:08.242502495Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-05T17:58:08.244777673Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:08.248322171Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-08-05T17:58:08.325720719Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-08-05T17:58:08.474120262Z Wrote inspect data to must-gather.
2026-08-05T17:58:08.488713643Z Gathering data for ns/openshift-network-console...
2026-08-05T17:58:08.515016164Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:08.721366361Z INFO: Waiting for network log collection to complete ...
2026-08-05T17:58:08.721405648Z INFO: Waiting for ovnk database copies to complete ...
2026-08-05T17:58:08.721777723Z INFO: Copying ovnk databases complete.
2026-08-05T17:58:08.723564009Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-08-05T17:58:08.726085312Z ovnk_database_store/
2026-08-05T17:58:08.726133761Z ovnk_database_store/ovnkube-node-r69sp_sbdb
2026-08-05T17:58:08.751183670Z ovnk_database_store/ovnkube-node-r69sp_nbdb
2026-08-05T17:58:08.758217458Z ovnk_database_store/ovnkube-node-9hn4t_sbdb
2026-08-05T17:58:08.775278478Z ovnk_database_store/ovnkube-node-9hn4t_nbdb
2026-08-05T17:58:08.782010145Z INFO: Getting status from alertmanager-main-0
2026-08-05T17:58:08.782195142Z ovnk_database_store/ovnkube-node-pjgjr_sbdb
2026-08-05T17:58:08.785058179Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:08.788150596Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-08-05T17:58:08.802811155Z ovnk_database_store/ovnkube-node-pjgjr_nbdb
2026-08-05T17:58:08.814577173Z INFO: Network log collection complete.
2026-08-05T17:58:08.849968959Z Wrote inspect data to must-gather.
2026-08-05T17:58:09.016121255Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:09.030663116Z Gathering data for ns/openshift-console-operator...
2026-08-05T17:58:09.177778264Z Wrote inspect data to must-gather.
2026-08-05T17:58:09.232213071Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:09.235489552Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-08-05T17:58:09.444952725Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-08-05T17:58:09.447253578Z INFO: Fallback to identify the container image from release info
2026-08-05T17:58:09.454179871Z Gathering data for ns/openshift-console...
2026-08-05T17:58:09.473482244Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:09.671121351Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:09.673938502Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-08-05T17:58:09.848399775Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:09.996098867Z Gathering data for ns/openshift-cluster-storage-operator...
2026-08-05T17:58:10.046976632Z Wrote inspect data to /must-gather/istio.
2026-08-05T17:58:10.193089268Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-08-05T17:58:10.339403831Z daemonset.apps/perf-node-gather-daemonset created
2026-08-05T17:58:10.487009097Z Waiting for performance profile collector pods to become ready: 1
2026-08-05T17:58:10.566257939Z Gathering data for ns/openshift-dns-operator...
2026-08-05T17:58:10.816108813Z Done executing Istio gather script
2026-08-05T17:58:10.911701480Z Gathering data for ns/openshift-dns...
2026-08-05T17:58:11.115033431Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-08-05T17:58:11.500847832Z Gathering data for ns/openshift-image-registry...
2026-08-05T17:58:11.593637107Z Waiting for performance profile collector pods to become ready: 2
2026-08-05T17:58:11.990396608Z Gathering data for ns/openshift-ingress-operator...
2026-08-05T17:58:12.304045559Z Gathering data for ns/openshift-ingress...
2026-08-05T17:58:12.710478446Z Gathering data for ns/openshift-ingress-canary...
2026-08-05T17:58:12.719702883Z Waiting for performance profile collector pods to become ready: 3
2026-08-05T17:58:13.103932769Z Gathering data for ns/openshift-insights...
2026-08-05T17:58:13.839673208Z Waiting for performance profile collector pods to become ready: 4
2026-08-05T17:58:14.967030386Z Waiting for performance profile collector pods to become ready: 5
2026-08-05T17:58:14.995381900Z Gathering data for ns/openshift-jobset-operator...
2026-08-05T17:58:15.768814890Z Gathering data for ns/openshift-config...
2026-08-05T17:58:16.081607356Z Gathering data for ns/openshift-config-managed...
2026-08-05T17:58:16.096693399Z Waiting for performance profile collector pods to become ready: 6
2026-08-05T17:58:16.121320023Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-08-05T17:58:16.490624712Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-08-05T17:58:16.750993384Z Gathering data for ns/openshift-kube-apiserver...
2026-08-05T17:58:17.017374825Z Gathering data for ns/openshift-kube-controller-manager...
2026-08-05T17:58:17.211996957Z Waiting for performance profile collector pods to become ready: 7
2026-08-05T17:58:17.330283098Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-08-05T17:58:17.611748965Z Gathering data for ns/openshift-kube-scheduler...
2026-08-05T17:58:17.910203122Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-08-05T17:58:18.201601427Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-08-05T17:58:18.332622040Z Waiting for performance profile collector pods to become ready: 8
2026-08-05T17:58:18.529057526Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-08-05T17:58:18.831700501Z Gathering data for ns/openshift-user-workload-monitoring...
2026-08-05T17:58:19.311488005Z Gathering data for ns/openshift-multus...
2026-08-05T17:58:19.437354644Z Waiting for performance profile collector pods to become ready: 9
2026-08-05T17:58:20.565042871Z Waiting for performance profile collector pods to become ready: 10
2026-08-05T17:58:20.791855992Z Gathering data for ns/openshift-ovn-kubernetes...
2026-08-05T17:58:21.126848448Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-08-05T17:58:21.680769999Z Waiting for performance profile collector pods to become ready: 11
2026-08-05T17:58:22.060910875Z Gathering data for ns/openshift-host-network...
2026-08-05T17:58:22.391196759Z Gathering data for ns/openshift-network-diagnostics...
2026-08-05T17:58:22.810303281Z Waiting for performance profile collector pods to become ready: 12
2026-08-05T17:58:22.913227138Z Gathering data for ns/openshift-network-node-identity...
2026-08-05T17:58:23.362917346Z Gathering data for ns/openshift-network-operator...
2026-08-05T17:58:23.747058941Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-08-05T17:58:23.922638221Z Waiting for performance profile collector pods to become ready: 13
2026-08-05T17:58:24.046135109Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-08-05T17:58:24.525441542Z Gathering data for ns/openshift-apiserver-operator...
2026-08-05T17:58:24.811851502Z Gathering data for ns/openshift-apiserver...
2026-08-05T17:58:25.056496426Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-08-05T17:58:25.252542763Z Gathering data for ns/openshift-controller-manager-operator...
2026-08-05T17:58:25.336838465Z Collecting performance related data for node ip-10-0-143-121.ec2.internal
2026-08-05T17:58:25.344319474Z Collecting performance related data for node ip-10-0-135-56.ec2.internal
2026-08-05T17:58:25.352453372Z Collecting performance related data for node ip-10-0-139-158.ec2.internal
2026-08-05T17:58:25.560380116Z Gathering data for ns/openshift-controller-manager...
2026-08-05T17:58:25.874074123Z Gathering data for ns/openshift-cluster-samples-operator...
2026-08-05T17:58:26.132114001Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-08-05T17:58:26.713746828Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-08-05T17:58:27.128329157Z Gathering data for ns/openshift-service-ca-operator...
2026-08-05T17:58:27.472456782Z Gathering data for ns/openshift-service-ca...
2026-08-05T17:58:27.910951292Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-08-05T17:58:28.913168353Z Wrote inspect data to must-gather.
2026-08-05T17:58:28.913217566Z error: inspection completed with the errors occurred while gathering data:
2026-08-05T17:58:28.913217566Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-08-05T17:58:31.136940650Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-08-05T17:58:33.416081596Z Collecting kubelet logs for node ip-10-0-143-121.ec2.internal
2026-08-05T17:58:33.418010440Z Collecting kubelet logs for node ip-10-0-135-56.ec2.internal
2026-08-05T17:58:33.419843878Z Collecting kubelet logs for node ip-10-0-139-158.ec2.internal
2026-08-05T17:58:33.744796740Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-s5ltq namespace
2026-08-05T17:58:33.750912824Z INFO: Node performance data collection complete.
2026-08-05T17:58:33.892160805Z error: the server doesn't have a resource type "clusters"
2026-08-05T17:58:34.223796505Z Caches written to disk