2026-07-24T13:46:41.970212930Z [disk usage checker] Started
2026-07-24T13:46:41.973780933Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T13:46:42.194878882Z Gathering data for ns/openshift-cluster-version...
2026-07-24T13:46:42.333574291Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T13:46:42.433789530Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T13:46:42.516779072Z Gathering data for ns/default...
2026-07-24T13:46:42.823516633Z Gathering data for ns/openshift...
2026-07-24T13:46:43.258295593Z Gathering data for ns/kube-system...
2026-07-24T13:46:44.072109325Z Gathering data for ns/openshift-etcd...
2026-07-24T13:46:44.422223843Z Wrote inspect data to must-gather.
2026-07-24T13:46:44.422259959Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T13:46:44.422259959Z     namespaces "assisted-installer" not found
2026-07-24T13:46:44.509592299Z Waiting on subprocesses to finish execution.
2026-07-24T13:46:44.520071084Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-24T13:46:44.522616055Z Executing Istio gather script
2026-07-24T13:46:44.529897614Z INFO: Gathering HAProxy config files
2026-07-24T13:46:44.557683778Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-24T13:46:44.575850208Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-24T13:46:44.587486329Z INFO: Collecting host service logs for crio
2026-07-24T13:46:44.587839996Z INFO: Collecting host service logs for kubelet
2026-07-24T13:46:44.588214041Z INFO: Collecting host service logs for rpm-ostreed
2026-07-24T13:46:44.588561887Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-24T13:46:44.588913064Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-24T13:46:44.589194280Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-24T13:46:44.589473009Z INFO: Collecting host service logs for NetworkManager
2026-07-24T13:46:44.589771721Z INFO: Collecting host service logs for openvswitch
2026-07-24T13:46:44.590046995Z INFO: Collecting host service logs for ovs-configuration
2026-07-24T13:46:44.590325258Z INFO: Collecting host service logs for ovsdb-server
2026-07-24T13:46:44.590603906Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-24T13:46:44.590974562Z INFO: Waiting for worker host service log collection to complete ...
2026-07-24T13:46:44.625800176Z INFO: Waiting for node performance related collection to complete ...
2026-07-24T13:46:44.972738312Z INFO: "metallb-operator" not detected. Skipping.
2026-07-24T13:46:45.266114227Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-24T13:46:45.328355515Z No resources found
2026-07-24T13:46:45.339327955Z No resources found in openshift-etcd namespace.
2026-07-24T13:46:45.501883448Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-24T13:46:45.503198913Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-24T13:46:45.570834762Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-24T13:46:45.570913518Z INFO: on-disk MachineConfig config collection complete.
2026-07-24T13:46:45.692773739Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-9bqn9
2026-07-24T13:46:45.692835956Z insights-runtime-extractor-9rjl4
2026-07-24T13:46:45.692865714Z insights-runtime-extractor-qd9q9
2026-07-24T13:46:45.692884630Z insights-runtime-extractor-xjvvp
2026-07-24T13:46:45.807096952Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-24T13:46:45.859929225Z INFO: Worker host service log collection to complete.
2026-07-24T13:46:45.983775106Z error: the server doesn't have a resource type "performanceprofile"
2026-07-24T13:46:45.984408801Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-24T13:46:45.986975580Z Wrote inspect data to must-gather.
2026-07-24T13:46:46.053162071Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-24T13:46:46.053162071Z See 'oc rsync -h' for help and examples
2026-07-24T13:46:46.138191859Z ERROR: No running kube-apiserver pods found
2026-07-24T13:46:46.162633431Z Wrote inspect data to must-gather.
2026-07-24T13:46:46.300932099Z No resources found
2026-07-24T13:46:46.686510673Z Wrote inspect data to must-gather.
2026-07-24T13:46:46.757424633Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-24T13:46:46.762997883Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T13:46:46.820049921Z Gathering data for ns/openshift-monitoring...
2026-07-24T13:46:46.907017362Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-24T13:46:46.910174717Z tar: Removing leading `/' from member names
2026-07-24T13:46:46.910686930Z Wrote inspect data to must-gather.
2026-07-24T13:46:46.916318644Z INFO: HAProxy config collection complete.
2026-07-24T13:46:46.973748353Z Wrote inspect data to must-gather.
2026-07-24T13:46:47.002496399Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T13:46:47.018001211Z Wrote inspect data to must-gather.
2026-07-24T13:46:47.026577418Z Wrote inspect data to must-gather.
2026-07-24T13:46:47.055279808Z INFO: 'previous-logs' folder not found on ip-10-0-137-69.ec2.internal, skipping...
2026-07-24T13:46:47.137101964Z error: the server doesn't have a resource type "machineconfigs"
2026-07-24T13:46:47.257036398Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-24T13:46:47.257556612Z Wrote inspect data to must-gather.
2026-07-24T13:46:47.299957470Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-24T13:46:47.309525929Z INFO: 'previous-logs' folder not found on ip-10-0-138-191.ec2.internal, skipping...
2026-07-24T13:46:47.535308905Z INFO: Getting rules from prometheus-k8s-0
2026-07-24T13:46:47.548506130Z Wrote inspect data to must-gather.
2026-07-24T13:46:47.554683411Z Wrote inspect data to must-gather.
2026-07-24T13:46:47.571556411Z INFO: 'previous-logs' folder not found on ip-10-0-140-48.ec2.internal, skipping...
2026-07-24T13:46:47.571595291Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-24T13:46:47.571595291Z INFO: Machine Config Daemon termination log collection complete.
2026-07-24T13:46:47.693228331Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-24T13:46:47.791783981Z Wrote inspect data to must-gather.
2026-07-24T13:46:47.894846760Z Wrote inspect data to must-gather.
2026-07-24T13:46:48.014299261Z INFO: Getting status/config from prometheus-k8s-0
2026-07-24T13:46:48.052078502Z INFO: INTERCONNECT MODE
2026-07-24T13:46:48.052078502Z INFO: Gathering ovn-kubernetes DBs
2026-07-24T13:46:48.108747021Z Wrote inspect data to must-gather.
2026-07-24T13:46:48.222003714Z INFO: Gathering OVN_Northbound from ovnkube-node-5n4wt...
2026-07-24T13:46:48.222195957Z INFO: Gathering OVN_Northbound from ovnkube-node-66ntf...
2026-07-24T13:46:48.222375800Z INFO: Gathering OVN_Northbound from ovnkube-node-j7z6j...
2026-07-24T13:46:48.222610469Z INFO: Gathering OVN_Southbound from ovnkube-node-5n4wt...
2026-07-24T13:46:48.222910353Z INFO: Gathering OVN_Southbound from ovnkube-node-66ntf...
2026-07-24T13:46:48.223267257Z INFO: Gathering OVN_Southbound from ovnkube-node-j7z6j...
2026-07-24T13:46:48.352660906Z Wrote inspect data to must-gather.
2026-07-24T13:46:48.403566841Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-24T13:46:48.531266957Z tar: Removing leading `/' from member names
2026-07-24T13:46:48.547154520Z tar: Removing leading `/' from member names
2026-07-24T13:46:48.556921308Z tar: Removing leading `/' from member names
2026-07-24T13:46:48.569249589Z tar: Removing leading `/' from member names
2026-07-24T13:46:48.571182605Z tar: Removing leading `/' from member names
2026-07-24T13:46:48.575729212Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-24T13:46:48.579525028Z tar: Removing leading `/' from member names
2026-07-24T13:46:48.627072215Z Wrote inspect data to must-gather.
2026-07-24T13:46:48.640047259Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:48.826559428Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-24T13:46:48.859032962Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:48.861807669Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-24T13:46:48.861980625Z Wrote inspect data to must-gather.
2026-07-24T13:46:49.037373210Z INFO: Waiting for network log collection to complete ...
2026-07-24T13:46:49.037409591Z INFO: Waiting for ovnk database copies to complete ...
2026-07-24T13:46:49.037818086Z INFO: Copying ovnk databases complete.
2026-07-24T13:46:49.039565226Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-24T13:46:49.042859083Z ovnk_database_store/
2026-07-24T13:46:49.042910945Z ovnk_database_store/ovnkube-node-j7z6j_sbdb
2026-07-24T13:46:49.057740750Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:49.059536004Z ovnk_database_store/ovnkube-node-5n4wt_sbdb
2026-07-24T13:46:49.074007946Z Wrote inspect data to must-gather.
2026-07-24T13:46:49.074414018Z ovnk_database_store/ovnkube-node-5n4wt_nbdb
2026-07-24T13:46:49.078601229Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-24T13:46:49.080495799Z ovnk_database_store/ovnkube-node-66ntf_nbdb
2026-07-24T13:46:49.086925344Z ovnk_database_store/ovnkube-node-66ntf_sbdb
2026-07-24T13:46:49.106310096Z ovnk_database_store/ovnkube-node-j7z6j_nbdb
2026-07-24T13:46:49.116761102Z INFO: Network log collection complete.
2026-07-24T13:46:49.290992220Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:49.294026278Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-24T13:46:49.313227943Z Wrote inspect data to must-gather.
2026-07-24T13:46:49.325258142Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T13:46:49.400151858Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-24T13:46:49.473666263Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:49.566844351Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-24T13:46:49.569360063Z INFO: Fallback to identify the container image from release info
2026-07-24T13:46:49.678343670Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:49.681479488Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-24T13:46:49.771716815Z INFO: Getting status from alertmanager-main-0
2026-07-24T13:46:49.898847952Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:49.933083549Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T13:46:50.086274767Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:50.088460612Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-24T13:46:50.108052487Z Gathering data for ns/openshift-network-console...
2026-07-24T13:46:50.196979058Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-24T13:46:50.259408829Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:50.318091419Z daemonset.apps/perf-node-gather-daemonset created
2026-07-24T13:46:50.455880276Z Waiting for performance profile collector pods to become ready: 1
2026-07-24T13:46:50.456100396Z Wrote inspect data to /must-gather/istio.
2026-07-24T13:46:50.601137859Z Gathering data for ns/openshift-console-operator...
2026-07-24T13:46:51.095742269Z Gathering data for ns/openshift-console...
2026-07-24T13:46:51.132089457Z Done executing Istio gather script
2026-07-24T13:46:51.573095315Z Waiting for performance profile collector pods to become ready: 2
2026-07-24T13:46:51.674096289Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-24T13:46:52.008686614Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T13:46:52.251919117Z Gathering data for ns/openshift-dns-operator...
2026-07-24T13:46:52.637966272Z Gathering data for ns/openshift-dns...
2026-07-24T13:46:52.680779307Z Waiting for performance profile collector pods to become ready: 3
2026-07-24T13:46:53.475750240Z Gathering data for ns/openshift-image-registry...
2026-07-24T13:46:53.782688267Z Waiting for performance profile collector pods to become ready: 4
2026-07-24T13:46:54.081681015Z Gathering data for ns/openshift-ingress-operator...
2026-07-24T13:46:54.466763835Z Gathering data for ns/openshift-ingress...
2026-07-24T13:46:54.900061287Z Waiting for performance profile collector pods to become ready: 5
2026-07-24T13:46:54.905068591Z Gathering data for ns/openshift-ingress-canary...
2026-07-24T13:46:55.442729441Z Gathering data for ns/openshift-insights...
2026-07-24T13:46:56.019074028Z Waiting for performance profile collector pods to become ready: 6
2026-07-24T13:46:57.013296885Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T13:46:57.130905783Z Waiting for performance profile collector pods to become ready: 7
2026-07-24T13:46:57.717839145Z Gathering data for ns/openshift-jobset-operator...
2026-07-24T13:46:58.266345756Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-24T13:46:58.496113724Z Collecting performance related data for node ip-10-0-138-191.ec2.internal
2026-07-24T13:46:58.503131111Z Collecting performance related data for node ip-10-0-140-48.ec2.internal
2026-07-24T13:46:58.518720563Z Collecting performance related data for node ip-10-0-137-69.ec2.internal
2026-07-24T13:46:58.658992310Z Gathering data for ns/openshift-config...
2026-07-24T13:46:59.173998767Z Gathering data for ns/openshift-config-managed...
2026-07-24T13:46:59.782923953Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-24T13:47:00.113830372Z Gathering data for ns/openshift-kube-apiserver...
2026-07-24T13:47:00.477435565Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-24T13:47:00.814186190Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-24T13:47:01.222791606Z Gathering data for ns/openshift-kube-scheduler...
2026-07-24T13:47:01.640670861Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-24T13:47:02.003773497Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-24T13:47:02.017917363Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T13:47:02.372204804Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-24T13:47:02.765419001Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-24T13:47:03.420036762Z Gathering data for ns/openshift-multus...
2026-07-24T13:47:05.306246275Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-24T13:47:06.863196804Z Collecting kubelet logs for node ip-10-0-138-191.ec2.internal
2026-07-24T13:47:06.865799341Z Collecting kubelet logs for node ip-10-0-140-48.ec2.internal
2026-07-24T13:47:06.868002109Z Collecting kubelet logs for node ip-10-0-137-69.ec2.internal
2026-07-24T13:47:07.024772067Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T13:47:07.140248167Z Gathering data for ns/openshift-host-network...
2026-07-24T13:47:07.233995165Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-tqt25 namespace
2026-07-24T13:47:07.239639226Z INFO: Node performance data collection complete.
2026-07-24T13:47:07.549087608Z Gathering data for ns/openshift-network-diagnostics...
2026-07-24T13:47:08.098086563Z Gathering data for ns/openshift-network-node-identity...
2026-07-24T13:47:08.562859860Z Gathering data for ns/openshift-network-operator...
2026-07-24T13:47:08.995474133Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-24T13:47:09.373065681Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-24T13:47:09.923716373Z Gathering data for ns/openshift-apiserver-operator...
2026-07-24T13:47:10.243064238Z Gathering data for ns/openshift-apiserver...
2026-07-24T13:47:10.647429353Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-24T13:47:10.977008701Z Gathering data for ns/openshift-controller-manager...
2026-07-24T13:47:11.257246473Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-24T13:47:11.897395995Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-24T13:47:12.029612719Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T13:47:12.248914265Z Gathering data for ns/openshift-service-ca-operator...
2026-07-24T13:47:12.620403204Z Gathering data for ns/openshift-service-ca...
2026-07-24T13:47:13.027910898Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-24T13:47:13.941465988Z Wrote inspect data to must-gather.
2026-07-24T13:47:13.941507652Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T13:47:13.941507652Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-24T13:47:14.066055421Z error: the server doesn't have a resource type "clusters"
2026-07-24T13:47:14.197133709Z Caches written to disk