2026-07-29T09:14:55.007646087Z [disk usage checker] Started
2026-07-29T09:14:55.010985095Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-29T09:14:55.210510319Z Gathering data for ns/openshift-cluster-version...
2026-07-29T09:14:55.353042093Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T09:14:55.436039173Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T09:14:55.524320137Z Gathering data for ns/default...
2026-07-29T09:14:55.799713291Z Gathering data for ns/openshift...
2026-07-29T09:14:56.190961058Z Gathering data for ns/kube-system...
2026-07-29T09:14:56.846992211Z Gathering data for ns/openshift-etcd...
2026-07-29T09:14:57.175725171Z Wrote inspect data to must-gather.
2026-07-29T09:14:57.175760967Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T09:14:57.175760967Z     namespaces "assisted-installer" not found
2026-07-29T09:14:57.587638922Z Waiting on subprocesses to finish execution.
2026-07-29T09:14:57.594928486Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-29T09:14:57.600841297Z INFO: Gathering HAProxy config files
2026-07-29T09:14:57.605540556Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-29T09:14:57.607686434Z Executing Istio gather script
2026-07-29T09:14:57.612610402Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-29T09:14:57.624927403Z INFO: Collecting host service logs for crio
2026-07-29T09:14:57.625298512Z INFO: Collecting host service logs for kubelet
2026-07-29T09:14:57.625682902Z INFO: Collecting host service logs for rpm-ostreed
2026-07-29T09:14:57.626089212Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-29T09:14:57.626458916Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-29T09:14:57.626817564Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-29T09:14:57.627159818Z INFO: Collecting host service logs for NetworkManager
2026-07-29T09:14:57.627547528Z INFO: Collecting host service logs for openvswitch
2026-07-29T09:14:57.627885560Z INFO: Collecting host service logs for ovs-configuration
2026-07-29T09:14:57.628239272Z INFO: Collecting host service logs for ovsdb-server
2026-07-29T09:14:57.628612952Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-29T09:14:57.629016734Z INFO: Waiting for worker host service log collection to complete ...
2026-07-29T09:14:57.749007692Z INFO: Waiting for node performance related collection to complete ...
2026-07-29T09:14:58.115343740Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-29T09:14:58.174753281Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-29T09:14:58.174805289Z INFO: on-disk MachineConfig config collection complete.
2026-07-29T09:14:58.199076404Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-2zvkv
2026-07-29T09:14:58.199076404Z insights-runtime-extractor-82859
2026-07-29T09:14:58.199076404Z insights-runtime-extractor-9xgnc
2026-07-29T09:14:58.199076404Z insights-runtime-extractor-q7zcn
2026-07-29T09:14:58.334459549Z No resources found
2026-07-29T09:14:58.421144508Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-29T09:14:58.473647797Z No resources found in openshift-etcd namespace.
2026-07-29T09:14:58.651185228Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-29T09:14:58.681026369Z Wrote inspect data to must-gather.
2026-07-29T09:14:58.689676266Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-29T09:14:58.734584730Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-29T09:14:58.734584730Z See 'oc rsync -h' for help and examples
2026-07-29T09:14:58.939939999Z ERROR: No running kube-apiserver pods found
2026-07-29T09:14:59.049488887Z INFO: "metallb-operator" not detected. Skipping.
2026-07-29T09:14:59.134799451Z error: the server doesn't have a resource type "performanceprofile"
2026-07-29T09:14:59.236652534Z Wrote inspect data to must-gather.
2026-07-29T09:14:59.251771568Z INFO: Worker host service log collection to complete.
2026-07-29T09:14:59.490504689Z Wrote inspect data to must-gather.
2026-07-29T09:14:59.706961967Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-29T09:14:59.716161879Z INFO: 'previous-logs' folder not found on ip-10-0-135-133.ec2.internal, skipping...
2026-07-29T09:14:59.846665482Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T09:14:59.863864261Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-29T09:14:59.906838025Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-29T09:14:59.963655150Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.042110962Z No resources found
2026-07-29T09:15:00.044801458Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.046707755Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-29T09:15:00.062879188Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.152613915Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.157883734Z Gathering data for ns/openshift-monitoring...
2026-07-29T09:15:00.244032889Z INFO: 'previous-logs' folder not found on ip-10-0-137-122.ec2.internal, skipping...
2026-07-29T09:15:00.306220308Z tar: Removing leading `/' from member names
2026-07-29T09:15:00.312725198Z INFO: HAProxy config collection complete.
2026-07-29T09:15:00.341987543Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.395565323Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-29T09:15:00.399667599Z error: the server doesn't have a resource type "machineconfigs"
2026-07-29T09:15:00.454678787Z INFO: Getting rules from prometheus-k8s-0
2026-07-29T09:15:00.457907244Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.548862087Z INFO: 'previous-logs' folder not found on ip-10-0-140-143.ec2.internal, skipping...
2026-07-29T09:15:00.548893959Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-29T09:15:00.548902654Z INFO: Machine Config Daemon termination log collection complete.
2026-07-29T09:15:00.573982415Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-29T09:15:00.712419933Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.778269556Z Wrote inspect data to must-gather.
2026-07-29T09:15:00.847975739Z INFO: Getting status/config from prometheus-k8s-0
2026-07-29T09:15:00.941164987Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-29T09:15:00.999819086Z INFO: INTERCONNECT MODE
2026-07-29T09:15:00.999819086Z INFO: Gathering ovn-kubernetes DBs
2026-07-29T09:15:01.150116843Z Wrote inspect data to must-gather.
2026-07-29T09:15:01.155964647Z INFO: Gathering OVN_Northbound from ovnkube-node-q4pjp...
2026-07-29T09:15:01.156255181Z INFO: Gathering OVN_Northbound from ovnkube-node-qrgkd...
2026-07-29T09:15:01.156600500Z INFO: Gathering OVN_Northbound from ovnkube-node-sj7s2...
2026-07-29T09:15:01.157047164Z INFO: Gathering OVN_Southbound from ovnkube-node-q4pjp...
2026-07-29T09:15:01.157342733Z INFO: Gathering OVN_Southbound from ovnkube-node-qrgkd...
2026-07-29T09:15:01.157652892Z INFO: Gathering OVN_Southbound from ovnkube-node-sj7s2...
2026-07-29T09:15:01.359933940Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-29T09:15:01.421849788Z tar: Removing leading `/' from member names
2026-07-29T09:15:01.437770881Z tar: Removing leading `/' from member names
2026-07-29T09:15:01.472896482Z Wrote inspect data to must-gather.
2026-07-29T09:15:01.484677259Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-29T09:15:01.486430119Z tar: Removing leading `/' from member names
2026-07-29T09:15:01.494625956Z tar: Removing leading `/' from member names
2026-07-29T09:15:01.499698180Z tar: Removing leading `/' from member names
2026-07-29T09:15:01.545609057Z tar: Removing leading `/' from member names
2026-07-29T09:15:01.650661483Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-29T09:15:01.695285310Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T09:15:01.719482024Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:01.744070003Z Wrote inspect data to must-gather.
2026-07-29T09:15:01.920446118Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-29T09:15:01.983004772Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:01.987601505Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-29T09:15:02.015818552Z Wrote inspect data to must-gather.
2026-07-29T09:15:02.047360295Z INFO: Waiting for network log collection to complete ...
2026-07-29T09:15:02.047514819Z INFO: Waiting for ovnk database copies to complete ...
2026-07-29T09:15:02.048052496Z INFO: Copying ovnk databases complete.
2026-07-29T09:15:02.052479923Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-29T09:15:02.055181144Z ovnk_database_store/
2026-07-29T09:15:02.055229754Z ovnk_database_store/ovnkube-node-q4pjp_sbdb
2026-07-29T09:15:02.071414594Z ovnk_database_store/ovnkube-node-sj7s2_nbdb
2026-07-29T09:15:02.077790261Z ovnk_database_store/ovnkube-node-q4pjp_nbdb
2026-07-29T09:15:02.084312701Z ovnk_database_store/ovnkube-node-qrgkd_nbdb
2026-07-29T09:15:02.091821918Z ovnk_database_store/ovnkube-node-sj7s2_sbdb
2026-07-29T09:15:02.110722603Z ovnk_database_store/ovnkube-node-qrgkd_sbdb
2026-07-29T09:15:02.130606003Z INFO: Network log collection complete.
2026-07-29T09:15:02.182597960Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-29T09:15:02.208225657Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:02.227981143Z W0729 09:15:02.227928    1496 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-29T09:15:02.235278938Z Wrote inspect data to must-gather.
2026-07-29T09:15:02.333573149Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T09:15:02.442942440Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:02.446597837Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-29T09:15:02.542633409Z Gathering data for ns/openshift-network-console...
2026-07-29T09:15:02.547509957Z Wrote inspect data to must-gather.
2026-07-29T09:15:02.589063158Z INFO: Getting status from alertmanager-main-0
2026-07-29T09:15:02.670712667Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:02.858370239Z Wrote inspect data to must-gather.
2026-07-29T09:15:02.886603573Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:02.890928543Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-29T09:15:03.007772550Z Gathering data for ns/openshift-console-operator...
2026-07-29T09:15:03.131799051Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:03.146734899Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-29T09:15:03.148812557Z INFO: Fallback to identify the container image from release info
2026-07-29T09:15:03.319320814Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:03.322968338Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-29T09:15:03.402351387Z Gathering data for ns/openshift-console...
2026-07-29T09:15:03.476178995Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:03.662470187Z Wrote inspect data to /must-gather/istio.
2026-07-29T09:15:03.916189512Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-29T09:15:03.932745351Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-29T09:15:04.068026196Z daemonset.apps/perf-node-gather-daemonset created
2026-07-29T09:15:04.204989446Z Waiting for performance profile collector pods to become ready: 1
2026-07-29T09:15:04.369705395Z Done executing Istio gather script
2026-07-29T09:15:04.454199487Z Gathering data for ns/openshift-dns-operator...
2026-07-29T09:15:04.766034627Z Gathering data for ns/openshift-dns...
2026-07-29T09:15:05.051754682Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-29T09:15:05.314784182Z Waiting for performance profile collector pods to become ready: 2
2026-07-29T09:15:05.571286589Z Gathering data for ns/openshift-image-registry...
2026-07-29T09:15:06.083582925Z Gathering data for ns/openshift-ingress-operator...
2026-07-29T09:15:06.428939098Z Gathering data for ns/openshift-ingress...
2026-07-29T09:15:06.442983375Z Waiting for performance profile collector pods to become ready: 3
2026-07-29T09:15:06.754777524Z Gathering data for ns/openshift-ingress-canary...
2026-07-29T09:15:07.153333100Z Gathering data for ns/openshift-insights...
2026-07-29T09:15:07.548371676Z Waiting for performance profile collector pods to become ready: 4
2026-07-29T09:15:08.656281074Z Waiting for performance profile collector pods to become ready: 5
2026-07-29T09:15:09.086997294Z Gathering data for ns/openshift-jobset-operator...
2026-07-29T09:15:09.760531904Z Waiting for performance profile collector pods to become ready: 6
2026-07-29T09:15:09.855032542Z Gathering data for ns/openshift-config...
2026-07-29T09:15:10.057569614Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-29T09:15:10.106491940Z Gathering data for ns/openshift-config-managed...
2026-07-29T09:15:10.489152094Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-29T09:15:10.775673625Z Gathering data for ns/openshift-kube-apiserver...
2026-07-29T09:15:10.883025651Z Waiting for performance profile collector pods to become ready: 7
2026-07-29T09:15:11.026112264Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-29T09:15:11.341916632Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-29T09:15:11.620837456Z Gathering data for ns/openshift-kube-scheduler...
2026-07-29T09:15:11.912685465Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-29T09:15:12.013042472Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-29T09:15:12.209599809Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-29T09:15:12.268504394Z Collecting performance related data for node ip-10-0-137-122.ec2.internal
2026-07-29T09:15:12.278319450Z Collecting performance related data for node ip-10-0-140-143.ec2.internal
2026-07-29T09:15:12.287513492Z Collecting performance related data for node ip-10-0-135-133.ec2.internal
2026-07-29T09:15:12.565757379Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-29T09:15:12.942261151Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-29T09:15:13.579466007Z Gathering data for ns/openshift-multus...
2026-07-29T09:15:15.005120842Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-29T09:15:15.065036219Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-29T09:15:16.203203208Z Gathering data for ns/openshift-host-network...
2026-07-29T09:15:16.506308243Z Gathering data for ns/openshift-network-diagnostics...
2026-07-29T09:15:17.039976935Z Gathering data for ns/openshift-network-node-identity...
2026-07-29T09:15:17.480541334Z Gathering data for ns/openshift-network-operator...
2026-07-29T09:15:17.851509302Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-29T09:15:18.101314442Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-29T09:15:18.598187368Z Gathering data for ns/openshift-apiserver-operator...
2026-07-29T09:15:18.872154275Z Gathering data for ns/openshift-apiserver...
2026-07-29T09:15:19.253246149Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-29T09:15:19.512193747Z Gathering data for ns/openshift-controller-manager...
2026-07-29T09:15:19.779534357Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-29T09:15:20.069881082Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-29T09:15:20.331648748Z Collecting kubelet logs for node ip-10-0-137-122.ec2.internal
2026-07-29T09:15:20.333288365Z Collecting kubelet logs for node ip-10-0-140-143.ec2.internal
2026-07-29T09:15:20.335105226Z Collecting kubelet logs for node ip-10-0-135-133.ec2.internal
2026-07-29T09:15:20.443865839Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-29T09:15:20.678400451Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-zmk9w namespace
2026-07-29T09:15:20.683321479Z INFO: Node performance data collection complete.
2026-07-29T09:15:20.748928320Z Gathering data for ns/openshift-service-ca-operator...
2026-07-29T09:15:21.042810525Z Gathering data for ns/openshift-service-ca...
2026-07-29T09:15:21.373753863Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-29T09:15:22.216458034Z Wrote inspect data to must-gather.
2026-07-29T09:15:22.216648421Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T09:15:22.216648421Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-29T09:15:22.341649009Z error: the server doesn't have a resource type "clusters"
2026-07-29T09:15:22.472295309Z Caches written to disk