2026-09-07T10:40:11.135053243Z [disk usage checker] Started
2026-09-07T10:40:11.138315884Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-07T10:40:11.336661915Z Gathering data for ns/openshift-cluster-version...
2026-09-07T10:40:11.475054858Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T10:40:11.555362887Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T10:40:11.632668774Z Gathering data for ns/default...
2026-09-07T10:40:11.902452773Z Gathering data for ns/openshift...
2026-09-07T10:40:12.285639777Z Gathering data for ns/kube-system...
2026-09-07T10:40:13.027868880Z Gathering data for ns/openshift-etcd...
2026-09-07T10:40:13.322241296Z Wrote inspect data to must-gather.
2026-09-07T10:40:13.322313603Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T10:40:13.322313603Z     namespaces "assisted-installer" not found
2026-09-07T10:40:13.718680964Z Waiting on subprocesses to finish execution.
2026-09-07T10:40:13.728124575Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-07T10:40:13.738532805Z Executing Istio gather script
2026-09-07T10:40:13.738758527Z INFO: Gathering HAProxy config files
2026-09-07T10:40:13.739517823Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-07T10:40:13.749650660Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-07T10:40:13.757086489Z INFO: Collecting host service logs for crio
2026-09-07T10:40:13.757415314Z INFO: Collecting host service logs for kubelet
2026-09-07T10:40:13.757746615Z INFO: Collecting host service logs for rpm-ostreed
2026-09-07T10:40:13.758058387Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-07T10:40:13.758374611Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-07T10:40:13.758687265Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-07T10:40:13.759010279Z INFO: Collecting host service logs for NetworkManager
2026-09-07T10:40:13.759317664Z INFO: Collecting host service logs for openvswitch
2026-09-07T10:40:13.759644424Z INFO: Collecting host service logs for ovs-configuration
2026-09-07T10:40:13.759960939Z INFO: Collecting host service logs for ovsdb-server
2026-09-07T10:40:13.760272081Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-07T10:40:13.760638285Z INFO: Waiting for worker host service log collection to complete ...
2026-09-07T10:40:13.830529035Z INFO: Waiting for node performance related collection to complete ...
2026-09-07T10:40:14.225998415Z INFO: "metallb-operator" not detected. Skipping.
2026-09-07T10:40:14.279081246Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-07T10:40:14.520369046Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-78ld2
2026-09-07T10:40:14.520418137Z insights-runtime-extractor-9klwx
2026-09-07T10:40:14.520438361Z insights-runtime-extractor-gv4pf
2026-09-07T10:40:14.520457508Z insights-runtime-extractor-qrfzw
2026-09-07T10:40:14.614407034Z ERROR: No running kube-apiserver pods found
2026-09-07T10:40:14.711035931Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-07T10:40:14.711138265Z INFO: on-disk MachineConfig config collection complete.
2026-09-07T10:40:14.819866784Z Wrote inspect data to must-gather.
2026-09-07T10:40:14.836560672Z No resources found in openshift-etcd namespace.
2026-09-07T10:40:14.876487763Z No resources found
2026-09-07T10:40:14.880605426Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-07T10:40:15.175567227Z error: the server doesn't have a resource type "performanceprofile"
2026-09-07T10:40:15.193963648Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-07T10:40:15.204714502Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-07T10:40:15.274389074Z INFO: Worker host service log collection to complete.
2026-09-07T10:40:15.281026865Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-07T10:40:15.281026865Z See 'oc rsync -h' for help and examples
2026-09-07T10:40:15.796825150Z Wrote inspect data to must-gather.
2026-09-07T10:40:15.850285841Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-07T10:40:15.872323839Z Wrote inspect data to must-gather.
2026-09-07T10:40:15.885229855Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T10:40:15.911316855Z Wrote inspect data to must-gather.
2026-09-07T10:40:15.964923162Z No resources found
2026-09-07T10:40:16.027412788Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-07T10:40:16.140741558Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-07T10:40:16.142728367Z Wrote inspect data to must-gather.
2026-09-07T10:40:16.148152834Z Wrote inspect data to must-gather.
2026-09-07T10:40:16.171900431Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-07T10:40:16.191324141Z Gathering data for ns/openshift-monitoring...
2026-09-07T10:40:16.206064635Z INFO: 'previous-logs' folder not found on ip-10-0-131-125.ec2.internal, skipping...
2026-09-07T10:40:16.307443702Z Wrote inspect data to must-gather.
2026-09-07T10:40:16.368013574Z error: the server doesn't have a resource type "machineconfigs"
2026-09-07T10:40:16.402130306Z tar: Removing leading `/' from member names
2026-09-07T10:40:16.406625796Z INFO: HAProxy config collection complete.
2026-09-07T10:40:16.431480211Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-07T10:40:16.546112226Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-07T10:40:16.548424013Z INFO: 'previous-logs' folder not found on ip-10-0-140-225.ec2.internal, skipping...
2026-09-07T10:40:16.616679122Z Wrote inspect data to must-gather.
2026-09-07T10:40:16.637177378Z INFO: Getting rules from prometheus-k8s-0
2026-09-07T10:40:16.793527929Z Wrote inspect data to must-gather.
2026-09-07T10:40:16.833718945Z INFO: 'previous-logs' folder not found on ip-10-0-141-85.ec2.internal, skipping...
2026-09-07T10:40:16.833845629Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-07T10:40:16.833865882Z INFO: Machine Config Daemon termination log collection complete.
2026-09-07T10:40:16.874621387Z Wrote inspect data to must-gather.
2026-09-07T10:40:16.950034166Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-07T10:40:17.063969965Z INFO: Getting status/config from prometheus-k8s-0
2026-09-07T10:40:17.124697068Z Wrote inspect data to must-gather.
2026-09-07T10:40:17.161059588Z Wrote inspect data to must-gather.
2026-09-07T10:40:17.357594908Z Wrote inspect data to must-gather.
2026-09-07T10:40:17.406217658Z INFO: INTERCONNECT MODE
2026-09-07T10:40:17.406248440Z INFO: Gathering ovn-kubernetes DBs
2026-09-07T10:40:17.479206112Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-07T10:40:17.507997994Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-07T10:40:17.586989825Z Wrote inspect data to must-gather.
2026-09-07T10:40:17.591216781Z INFO: Gathering OVN_Northbound from ovnkube-node-4z57z...
2026-09-07T10:40:17.591593557Z INFO: Gathering OVN_Northbound from ovnkube-node-f6mnj...
2026-09-07T10:40:17.591867720Z INFO: Gathering OVN_Northbound from ovnkube-node-lcshb...
2026-09-07T10:40:17.592145136Z INFO: Gathering OVN_Southbound from ovnkube-node-4z57z...
2026-09-07T10:40:17.592430210Z INFO: Gathering OVN_Southbound from ovnkube-node-f6mnj...
2026-09-07T10:40:17.592721812Z INFO: Gathering OVN_Southbound from ovnkube-node-lcshb...
2026-09-07T10:40:17.804132822Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T10:40:17.811860663Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-07T10:40:17.845652889Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:17.905007215Z tar: Removing leading `/' from member names
2026-09-07T10:40:17.936712678Z Wrote inspect data to must-gather.
2026-09-07T10:40:17.945242913Z tar: Removing leading `/' from member names
2026-09-07T10:40:17.972691302Z tar: Removing leading `/' from member names
2026-09-07T10:40:17.973763670Z tar: Removing leading `/' from member names
2026-09-07T10:40:17.978838713Z tar: Removing leading `/' from member names
2026-09-07T10:40:18.002311551Z tar: Removing leading `/' from member names
2026-09-07T10:40:18.121997015Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:18.124820192Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-07T10:40:18.132267757Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-07T10:40:18.216917854Z Wrote inspect data to must-gather.
2026-09-07T10:40:18.377108230Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:18.407371316Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-07T10:40:18.461155679Z W0907 10:40:18.461087    1480 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-07T10:40:18.468672231Z Wrote inspect data to must-gather.
2026-09-07T10:40:18.512856694Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T10:40:18.517997330Z INFO: Waiting for network log collection to complete ...
2026-09-07T10:40:18.518107557Z INFO: Waiting for ovnk database copies to complete ...
2026-09-07T10:40:18.518618151Z INFO: Copying ovnk databases complete.
2026-09-07T10:40:18.520750785Z 3.6M	must-gather/network_logs/ovnk_database_store
2026-09-07T10:40:18.527835898Z ovnk_database_store/
2026-09-07T10:40:18.527893349Z ovnk_database_store/ovnkube-node-lcshb_nbdb
2026-09-07T10:40:18.538696788Z ovnk_database_store/ovnkube-node-4z57z_sbdb
2026-09-07T10:40:18.564019473Z ovnk_database_store/ovnkube-node-f6mnj_nbdb
2026-09-07T10:40:18.574044383Z ovnk_database_store/ovnkube-node-f6mnj_sbdb
2026-09-07T10:40:18.598125484Z ovnk_database_store/ovnkube-node-lcshb_sbdb
2026-09-07T10:40:18.619612393Z ovnk_database_store/ovnkube-node-4z57z_nbdb
2026-09-07T10:40:18.626848067Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:18.629909395Z INFO: Network log collection complete.
2026-09-07T10:40:18.630197887Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-07T10:40:18.759314594Z Wrote inspect data to must-gather.
2026-09-07T10:40:18.772714668Z INFO: Getting status from alertmanager-main-0
2026-09-07T10:40:18.803479367Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:18.928844312Z Gathering data for ns/openshift-network-console...
2026-09-07T10:40:19.020355315Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:19.023911916Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-07T10:40:19.034729611Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-07T10:40:19.037566092Z INFO: Fallback to identify the container image from release info
2026-09-07T10:40:19.240731533Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:19.326606732Z Gathering data for ns/openshift-console-operator...
2026-09-07T10:40:19.439752390Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:19.441897440Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-07T10:40:19.591171108Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:19.706269655Z Gathering data for ns/openshift-console...
2026-09-07T10:40:19.758973667Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-07T10:40:19.798745420Z Wrote inspect data to /must-gather/istio.
2026-09-07T10:40:19.904736817Z daemonset.apps/perf-node-gather-daemonset created
2026-09-07T10:40:20.049720907Z Waiting for performance profile collector pods to become ready: 1
2026-09-07T10:40:20.254498250Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-07T10:40:20.501107446Z Done executing Istio gather script
2026-09-07T10:40:20.739040932Z Gathering data for ns/openshift-dns-operator...
2026-09-07T10:40:20.992722018Z Gathering data for ns/openshift-dns...
2026-09-07T10:40:21.152912211Z Waiting for performance profile collector pods to become ready: 2
2026-09-07T10:40:21.177428477Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-07T10:40:21.571252462Z Gathering data for ns/openshift-image-registry...
2026-09-07T10:40:22.003824100Z Gathering data for ns/openshift-ingress-operator...
2026-09-07T10:40:22.281370630Z Waiting for performance profile collector pods to become ready: 3
2026-09-07T10:40:22.307747450Z Gathering data for ns/openshift-ingress...
2026-09-07T10:40:22.601235438Z Gathering data for ns/openshift-ingress-canary...
2026-09-07T10:40:22.939401644Z Gathering data for ns/openshift-insights...
2026-09-07T10:40:23.405852453Z Waiting for performance profile collector pods to become ready: 4
2026-09-07T10:40:24.516309816Z Waiting for performance profile collector pods to become ready: 5
2026-09-07T10:40:24.896106258Z Gathering data for ns/openshift-config...
2026-09-07T10:40:25.175483947Z Gathering data for ns/openshift-config-managed...
2026-09-07T10:40:25.575245804Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-07T10:40:25.647740596Z Waiting for performance profile collector pods to become ready: 6
2026-09-07T10:40:25.825839988Z Gathering data for ns/openshift-kube-apiserver...
2026-09-07T10:40:26.055518590Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-07T10:40:26.183066681Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-07T10:40:26.301907383Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-07T10:40:26.552936279Z Gathering data for ns/openshift-kube-scheduler...
2026-09-07T10:40:26.768856632Z Waiting for performance profile collector pods to become ready: 7
2026-09-07T10:40:26.794743248Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-07T10:40:27.114601645Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-07T10:40:27.399441180Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-07T10:40:27.704670750Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-07T10:40:27.884217337Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-07T10:40:28.129420350Z Collecting performance related data for node ip-10-0-141-85.ec2.internal
2026-09-07T10:40:28.136981832Z Collecting performance related data for node ip-10-0-140-225.ec2.internal
2026-09-07T10:40:28.145665986Z Collecting performance related data for node ip-10-0-131-125.ec2.internal
2026-09-07T10:40:28.210424211Z Gathering data for ns/openshift-multus...
2026-09-07T10:40:29.672805187Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-07T10:40:30.867675718Z Gathering data for ns/openshift-host-network...
2026-09-07T10:40:31.179978082Z Gathering data for ns/openshift-network-diagnostics...
2026-09-07T10:40:31.188098158Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-07T10:40:31.665320263Z Gathering data for ns/openshift-network-node-identity...
2026-09-07T10:40:32.030045811Z Gathering data for ns/openshift-network-operator...
2026-09-07T10:40:32.397144314Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-07T10:40:32.635332675Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-07T10:40:33.116003272Z Gathering data for ns/openshift-apiserver-operator...
2026-09-07T10:40:33.401898282Z Gathering data for ns/openshift-apiserver...
2026-09-07T10:40:33.823139131Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-07T10:40:34.075437258Z Gathering data for ns/openshift-controller-manager...
2026-09-07T10:40:34.330401311Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-07T10:40:34.935384224Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-07T10:40:35.204207656Z Gathering data for ns/openshift-service-ca-operator...
2026-09-07T10:40:35.553311171Z Gathering data for ns/openshift-service-ca...
2026-09-07T10:40:35.913886703Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-07T10:40:36.175476869Z Collecting kubelet logs for node ip-10-0-141-85.ec2.internal
2026-09-07T10:40:36.177276784Z Collecting kubelet logs for node ip-10-0-140-225.ec2.internal
2026-09-07T10:40:36.178948193Z Collecting kubelet logs for node ip-10-0-131-125.ec2.internal
2026-09-07T10:40:36.193768095Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-07T10:40:36.537539454Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4plbr namespace
2026-09-07T10:40:36.542061885Z INFO: Node performance data collection complete.
2026-09-07T10:40:36.822709448Z Wrote inspect data to must-gather.
2026-09-07T10:40:36.822760785Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T10:40:36.822760785Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-07T10:40:36.969482781Z error: the server doesn't have a resource type "clusters"
2026-09-07T10:40:37.088958161Z Caches written to disk