2026-07-03T17:37:13.096551641Z [disk usage checker] Started
2026-07-03T17:37:13.100411282Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-03T17:37:13.295590617Z Gathering data for ns/openshift-cluster-version...
2026-07-03T17:37:13.440289291Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-03T17:37:13.530736855Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-03T17:37:13.612685016Z Gathering data for ns/default...
2026-07-03T17:37:13.900333798Z Gathering data for ns/openshift...
2026-07-03T17:37:14.309003938Z Gathering data for ns/kube-system...
2026-07-03T17:37:15.146605581Z Gathering data for ns/openshift-etcd...
2026-07-03T17:37:15.489070995Z Wrote inspect data to must-gather.
2026-07-03T17:37:15.489112825Z error: inspection completed with the errors occurred while gathering data:
2026-07-03T17:37:15.489112825Z     namespaces "assisted-installer" not found
2026-07-03T17:37:15.727507398Z Waiting on subprocesses to finish execution.
2026-07-03T17:37:15.733976893Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-03T17:37:15.734271486Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-03T17:37:15.752239295Z INFO: Gathering HAProxy config files
2026-07-03T17:37:15.752986353Z INFO: Collecting host service logs for crio
2026-07-03T17:37:15.753465195Z INFO: Collecting host service logs for kubelet
2026-07-03T17:37:15.753839331Z INFO: Collecting host service logs for rpm-ostreed
2026-07-03T17:37:15.754197754Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-03T17:37:15.754223000Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-03T17:37:15.754585161Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-03T17:37:15.754941831Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-03T17:37:15.755309498Z INFO: Collecting host service logs for NetworkManager
2026-07-03T17:37:15.755659304Z INFO: Collecting host service logs for openvswitch
2026-07-03T17:37:15.755993339Z INFO: Collecting host service logs for ovs-configuration
2026-07-03T17:37:15.756363421Z INFO: Collecting host service logs for ovsdb-server
2026-07-03T17:37:15.756716025Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-03T17:37:15.757123979Z INFO: Waiting for worker host service log collection to complete ...
2026-07-03T17:37:15.780745243Z Executing Istio gather script
2026-07-03T17:37:15.890827640Z INFO: Waiting for node performance related collection to complete ...
2026-07-03T17:37:16.403905882Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-lp66n
2026-07-03T17:37:16.403955896Z insights-runtime-extractor-5x8rd
2026-07-03T17:37:16.403976469Z insights-runtime-extractor-bx4p2
2026-07-03T17:37:16.403994879Z insights-runtime-extractor-l67mk
2026-07-03T17:37:16.424019324Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-03T17:37:16.424114363Z INFO: on-disk MachineConfig config collection complete.
2026-07-03T17:37:16.440742514Z No resources found in openshift-etcd namespace.
2026-07-03T17:37:16.560406597Z Wrote inspect data to must-gather.
2026-07-03T17:37:16.739921097Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-03T17:37:16.822659718Z error: the server doesn't have a resource type "performanceprofile"
2026-07-03T17:37:17.349638235Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-03T17:37:17.356823984Z No resources found
2026-07-03T17:37:17.362887988Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-03T17:37:17.375212471Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-03T17:37:17.375212471Z See 'oc rsync -h' for help and examples
2026-07-03T17:37:17.473576853Z No resources found
2026-07-03T17:37:17.520688741Z Wrote inspect data to must-gather.
2026-07-03T17:37:17.534538366Z Wrote inspect data to must-gather.
2026-07-03T17:37:17.580520081Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-03T17:37:17.599713314Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-03T17:37:17.625213962Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-03T17:37:17.631311142Z Wrote inspect data to must-gather.
2026-07-03T17:37:17.656764682Z INFO: "metallb-operator" not detected. Skipping.
2026-07-03T17:37:17.821574044Z INFO: Worker host service log collection to complete.
2026-07-03T17:37:17.853065968Z ERROR: No running kube-apiserver pods found
2026-07-03T17:37:17.873055098Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-03T17:37:17.905741269Z Wrote inspect data to must-gather.
2026-07-03T17:37:17.968208678Z Wrote inspect data to must-gather.
2026-07-03T17:37:18.014786274Z Wrote inspect data to must-gather.
2026-07-03T17:37:18.109548886Z INFO: 'previous-logs' folder not found on ip-10-0-128-236.ec2.internal, skipping...
2026-07-03T17:37:18.110618492Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-03T17:37:18.111381957Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-03T17:37:18.206299810Z tar: Removing leading `/' from member names
2026-07-03T17:37:18.210439187Z INFO: HAProxy config collection complete.
2026-07-03T17:37:18.216621271Z error: the server doesn't have a resource type "machineconfigs"
2026-07-03T17:37:18.364109394Z Wrote inspect data to must-gather.
2026-07-03T17:37:18.364529406Z Gathering data for ns/openshift-monitoring...
2026-07-03T17:37:18.449313138Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-03T17:37:18.461938757Z INFO: 'previous-logs' folder not found on ip-10-0-130-231.ec2.internal, skipping...
2026-07-03T17:37:18.481594357Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-03T17:37:18.615798057Z INFO: Getting rules from prometheus-k8s-0
2026-07-03T17:37:18.667837672Z Wrote inspect data to must-gather.
2026-07-03T17:37:18.703444856Z Wrote inspect data to must-gather.
2026-07-03T17:37:18.748729743Z INFO: 'previous-logs' folder not found on ip-10-0-135-249.ec2.internal, skipping...
2026-07-03T17:37:18.748832694Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-03T17:37:18.748887830Z INFO: Machine Config Daemon termination log collection complete.
2026-07-03T17:37:18.858339283Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-03T17:37:18.921761653Z Wrote inspect data to must-gather.
2026-07-03T17:37:19.042279360Z INFO: Getting status/config from prometheus-k8s-0
2026-07-03T17:37:19.079944983Z Wrote inspect data to must-gather.
2026-07-03T17:37:19.222947239Z INFO: INTERCONNECT MODE
2026-07-03T17:37:19.222985052Z INFO: Gathering ovn-kubernetes DBs
2026-07-03T17:37:19.289639884Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-03T17:37:19.295489898Z Wrote inspect data to must-gather.
2026-07-03T17:37:19.378778162Z INFO: Gathering OVN_Northbound from ovnkube-node-h24dh...
2026-07-03T17:37:19.379035617Z INFO: Gathering OVN_Northbound from ovnkube-node-rcjhx...
2026-07-03T17:37:19.379305502Z INFO: Gathering OVN_Northbound from ovnkube-node-vt8dx...
2026-07-03T17:37:19.379585896Z INFO: Gathering OVN_Southbound from ovnkube-node-h24dh...
2026-07-03T17:37:19.379828634Z INFO: Gathering OVN_Southbound from ovnkube-node-rcjhx...
2026-07-03T17:37:19.380154385Z INFO: Gathering OVN_Southbound from ovnkube-node-vt8dx...
2026-07-03T17:37:19.513652736Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-03T17:37:19.607396583Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:19.616505647Z Wrote inspect data to must-gather.
2026-07-03T17:37:19.680786867Z tar: Removing leading `/' from member names
2026-07-03T17:37:19.686561464Z tar: Removing leading `/' from member names
2026-07-03T17:37:19.690318835Z tar: Removing leading `/' from member names
2026-07-03T17:37:19.693219531Z tar: Removing leading `/' from member names
2026-07-03T17:37:19.769186435Z tar: Removing leading `/' from member names
2026-07-03T17:37:19.769443851Z tar: Removing leading `/' from member names
2026-07-03T17:37:19.832924130Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-03T17:37:19.851594573Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:19.855085860Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-03T17:37:19.870925337Z Wrote inspect data to must-gather.
2026-07-03T17:37:20.092629373Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:20.093075220Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-03T17:37:20.121512109Z Wrote inspect data to must-gather.
2026-07-03T17:37:20.180269104Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-03T17:37:20.186636639Z INFO: Waiting for network log collection to complete ...
2026-07-03T17:37:20.186747842Z INFO: Waiting for ovnk database copies to complete ...
2026-07-03T17:37:20.187270243Z INFO: Copying ovnk databases complete.
2026-07-03T17:37:20.189480142Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-03T17:37:20.192148776Z ovnk_database_store/
2026-07-03T17:37:20.192203409Z ovnk_database_store/ovnkube-node-vt8dx_nbdb
2026-07-03T17:37:20.199075136Z ovnk_database_store/ovnkube-node-vt8dx_sbdb
2026-07-03T17:37:20.217393029Z ovnk_database_store/ovnkube-node-h24dh_nbdb
2026-07-03T17:37:20.224417678Z ovnk_database_store/ovnkube-node-rcjhx_sbdb
2026-07-03T17:37:20.240878517Z ovnk_database_store/ovnkube-node-h24dh_sbdb
2026-07-03T17:37:20.258324486Z ovnk_database_store/ovnkube-node-rcjhx_nbdb
2026-07-03T17:37:20.268012776Z INFO: Network log collection complete.
2026-07-03T17:37:20.309802116Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:20.312107989Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-03T17:37:20.327315562Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-03T17:37:20.330508264Z Wrote inspect data to must-gather.
2026-07-03T17:37:20.511146052Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:20.543512649Z Wrote inspect data to must-gather.
2026-07-03T17:37:20.722065503Z INFO: Getting status from alertmanager-main-0
2026-07-03T17:37:20.726350900Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:20.729176527Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-03T17:37:20.822521168Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-03T17:37:20.825083135Z INFO: Fallback to identify the container image from release info
2026-07-03T17:37:20.847800810Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-03T17:37:20.962664679Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:21.024964504Z Gathering data for ns/openshift-network-console...
2026-07-03T17:37:21.154357999Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:21.156921404Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-03T17:37:21.311969115Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:21.447920436Z Gathering data for ns/openshift-console-operator...
2026-07-03T17:37:21.503583162Z Wrote inspect data to /must-gather/istio.
2026-07-03T17:37:21.540315781Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-03T17:37:21.660306773Z daemonset.apps/perf-node-gather-daemonset created
2026-07-03T17:37:21.803076276Z Waiting for performance profile collector pods to become ready: 1
2026-07-03T17:37:21.865153054Z Gathering data for ns/openshift-console...
2026-07-03T17:37:22.209624250Z Done executing Istio gather script
2026-07-03T17:37:22.372505466Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-03T17:37:22.849708604Z Gathering data for ns/openshift-dns-operator...
2026-07-03T17:37:22.936126616Z Waiting for performance profile collector pods to become ready: 2
2026-07-03T17:37:23.116956023Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-03T17:37:23.201372555Z Gathering data for ns/openshift-dns...
2026-07-03T17:37:23.808557263Z Gathering data for ns/openshift-image-registry...
2026-07-03T17:37:24.051302046Z Waiting for performance profile collector pods to become ready: 3
2026-07-03T17:37:24.286332157Z Gathering data for ns/openshift-ingress-operator...
2026-07-03T17:37:24.607903430Z Gathering data for ns/openshift-ingress...
2026-07-03T17:37:24.929712453Z Gathering data for ns/openshift-ingress-canary...
2026-07-03T17:37:25.165419374Z Waiting for performance profile collector pods to become ready: 4
2026-07-03T17:37:25.324785633Z Gathering data for ns/openshift-insights...
2026-07-03T17:37:26.271678566Z Waiting for performance profile collector pods to become ready: 5
2026-07-03T17:37:27.207035572Z Gathering data for ns/openshift-config...
2026-07-03T17:37:27.377250363Z Waiting for performance profile collector pods to become ready: 6
2026-07-03T17:37:27.515765369Z Gathering data for ns/openshift-config-managed...
2026-07-03T17:37:27.929993610Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-03T17:37:28.122495328Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-03T17:37:28.202414061Z Gathering data for ns/openshift-kube-apiserver...
2026-07-03T17:37:28.467566763Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-03T17:37:28.496890347Z Waiting for performance profile collector pods to become ready: 7
2026-07-03T17:37:28.755970383Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-03T17:37:29.040854956Z Gathering data for ns/openshift-kube-scheduler...
2026-07-03T17:37:29.323850120Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-03T17:37:29.626403310Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-03T17:37:29.675877572Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-03T17:37:29.872391890Z Collecting performance related data for node ip-10-0-130-231.ec2.internal
2026-07-03T17:37:29.879900791Z Collecting performance related data for node ip-10-0-128-236.ec2.internal
2026-07-03T17:37:29.889280885Z Collecting performance related data for node ip-10-0-135-249.ec2.internal
2026-07-03T17:37:30.058279174Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-03T17:37:30.424015609Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-03T17:37:31.016758745Z Gathering data for ns/openshift-multus...
2026-07-03T17:37:32.900409928Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-03T17:37:33.127943021Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-03T17:37:34.290949201Z Gathering data for ns/openshift-host-network...
2026-07-03T17:37:34.696888334Z Gathering data for ns/openshift-network-diagnostics...
2026-07-03T17:37:35.242853554Z Gathering data for ns/openshift-network-node-identity...
2026-07-03T17:37:35.681452926Z Gathering data for ns/openshift-network-operator...
2026-07-03T17:37:36.112527122Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-03T17:37:36.365101556Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-03T17:37:36.993211036Z Gathering data for ns/openshift-apiserver-operator...
2026-07-03T17:37:37.294343481Z Gathering data for ns/openshift-apiserver...
2026-07-03T17:37:37.687987343Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-03T17:37:37.779519888Z Collecting kubelet logs for node ip-10-0-130-231.ec2.internal
2026-07-03T17:37:37.781381099Z Collecting kubelet logs for node ip-10-0-128-236.ec2.internal
2026-07-03T17:37:37.783164386Z Collecting kubelet logs for node ip-10-0-135-249.ec2.internal
2026-07-03T17:37:37.986867186Z Gathering data for ns/openshift-controller-manager...
2026-07-03T17:37:38.122636269Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-92jtr namespace
2026-07-03T17:37:38.127384545Z INFO: Node performance data collection complete.
2026-07-03T17:37:38.133415799Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-03T17:37:38.290257488Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-03T17:37:38.942310351Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-03T17:37:39.235271257Z Gathering data for ns/openshift-service-ca-operator...
2026-07-03T17:37:39.547564157Z Gathering data for ns/openshift-service-ca...
2026-07-03T17:37:39.898921589Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-03T17:37:40.803112583Z Wrote inspect data to must-gather.
2026-07-03T17:37:40.803171899Z error: inspection completed with the errors occurred while gathering data:
2026-07-03T17:37:40.803171899Z     [skipping gathering secrets/support due to error: secrets "support" not found, skipping gathering daemonsets.apps/ovn-ipsec-containerized due to error: daemonsets.apps "ovn-ipsec-containerized" not found]
2026-07-03T17:37:40.928174621Z error: the server doesn't have a resource type "clusters"
2026-07-03T17:37:41.051825758Z Caches written to disk