2026-09-11T02:57:18.171665336Z [disk usage checker] Started
2026-09-11T02:57:18.174997436Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T02:57:18.374477666Z Gathering data for ns/openshift-cluster-version...
2026-09-11T02:57:18.517305258Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T02:57:18.597401085Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T02:57:18.681011432Z Gathering data for ns/default...
2026-09-11T02:57:18.963241568Z Gathering data for ns/openshift...
2026-09-11T02:57:19.367546037Z Gathering data for ns/kube-system...
2026-09-11T02:57:20.028067810Z Gathering data for ns/openshift-etcd...
2026-09-11T02:57:20.311950329Z Wrote inspect data to must-gather.
2026-09-11T02:57:20.312052145Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T02:57:20.312052145Z     namespaces "assisted-installer" not found
2026-09-11T02:57:20.714332991Z Waiting on subprocesses to finish execution.
2026-09-11T02:57:20.727764413Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-11T02:57:20.729271274Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-11T02:57:20.729941204Z Executing Istio gather script
2026-09-11T02:57:20.734449950Z INFO: Gathering HAProxy config files
2026-09-11T02:57:20.736355671Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-11T02:57:20.767160886Z INFO: Collecting host service logs for crio
2026-09-11T02:57:20.767590505Z INFO: Collecting host service logs for kubelet
2026-09-11T02:57:20.768029068Z INFO: Collecting host service logs for rpm-ostreed
2026-09-11T02:57:20.768423128Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-11T02:57:20.768873724Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-11T02:57:20.769205104Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-11T02:57:20.773193312Z INFO: Collecting host service logs for NetworkManager
2026-09-11T02:57:20.773546430Z INFO: Collecting host service logs for openvswitch
2026-09-11T02:57:20.773932082Z INFO: Collecting host service logs for ovs-configuration
2026-09-11T02:57:20.774286612Z INFO: Collecting host service logs for ovsdb-server
2026-09-11T02:57:20.774742192Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-11T02:57:20.775200354Z INFO: Waiting for worker host service log collection to complete ...
2026-09-11T02:57:20.953090287Z INFO: Waiting for node performance related collection to complete ...
2026-09-11T02:57:21.098006651Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-11T02:57:21.098050538Z INFO: on-disk MachineConfig config collection complete.
2026-09-11T02:57:21.416976593Z INFO: Collecting Insights Archives from insights-operator-5844b856db-mkhwj
2026-09-11T02:57:21.417054683Z insights-runtime-extractor-8chld
2026-09-11T02:57:21.417076786Z insights-runtime-extractor-qnrkm
2026-09-11T02:57:21.417096401Z insights-runtime-extractor-vsk2n
2026-09-11T02:57:21.435820052Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-11T02:57:21.452291331Z No resources found in openshift-etcd namespace.
2026-09-11T02:57:21.500579732Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-11T02:57:21.682395909Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-11T02:57:21.787793483Z Wrote inspect data to must-gather.
2026-09-11T02:57:21.795590255Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-11T02:57:22.280330902Z INFO: "metallb-operator" not detected. Skipping.
2026-09-11T02:57:22.299733427Z error: the server doesn't have a resource type "performanceprofile"
2026-09-11T02:57:22.378309094Z No resources found
2026-09-11T02:57:22.379945404Z INFO: Worker host service log collection to complete.
2026-09-11T02:57:22.426956011Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-11T02:57:22.629308055Z Wrote inspect data to must-gather.
2026-09-11T02:57:22.706785861Z Wrote inspect data to must-gather.
2026-09-11T02:57:22.767582081Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-11T02:57:22.767582081Z See 'oc rsync -h' for help and examples
2026-09-11T02:57:22.881089657Z Gathering data for ns/openshift-monitoring...
2026-09-11T02:57:22.936246784Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T02:57:22.939224753Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.047747512Z No resources found
2026-09-11T02:57:23.096647475Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-11T02:57:23.116190575Z ERROR: No running kube-apiserver pods found
2026-09-11T02:57:23.125028272Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.153590415Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-11T02:57:23.168934432Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.185044954Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T02:57:23.208643418Z INFO: 'previous-logs' folder not found on ip-10-0-132-61.ec2.internal, skipping...
2026-09-11T02:57:23.228823021Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.253080101Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-11T02:57:23.391353577Z tar: Removing leading `/' from member names
2026-09-11T02:57:23.395974161Z INFO: HAProxy config collection complete.
2026-09-11T02:57:23.415340167Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.422156340Z error: the server doesn't have a resource type "machineconfigs"
2026-09-11T02:57:23.503987992Z INFO: 'previous-logs' folder not found on ip-10-0-132-88.ec2.internal, skipping...
2026-09-11T02:57:23.510703066Z INFO: Getting rules from prometheus-k8s-0
2026-09-11T02:57:23.586777435Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-11T02:57:23.675588477Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.780526143Z INFO: 'previous-logs' folder not found on ip-10-0-138-210.ec2.internal, skipping...
2026-09-11T02:57:23.781785176Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-11T02:57:23.781859703Z INFO: Machine Config Daemon termination log collection complete.
2026-09-11T02:57:23.816785242Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.915919556Z INFO: Getting status/config from prometheus-k8s-0
2026-09-11T02:57:23.944449310Z Wrote inspect data to must-gather.
2026-09-11T02:57:23.977245107Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-11T02:57:24.202523144Z Wrote inspect data to must-gather.
2026-09-11T02:57:24.236508401Z INFO: INTERCONNECT MODE
2026-09-11T02:57:24.236508401Z INFO: Gathering ovn-kubernetes DBs
2026-09-11T02:57:24.295352040Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-11T02:57:24.345866074Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-11T02:57:24.436742719Z Wrote inspect data to must-gather.
2026-09-11T02:57:24.455983012Z INFO: Gathering OVN_Northbound from ovnkube-node-jv2sz...
2026-09-11T02:57:24.456331628Z INFO: Gathering OVN_Northbound from ovnkube-node-szlgt...
2026-09-11T02:57:24.456613147Z INFO: Gathering OVN_Northbound from ovnkube-node-xjmbk...
2026-09-11T02:57:24.456954917Z INFO: Gathering OVN_Southbound from ovnkube-node-jv2sz...
2026-09-11T02:57:24.457238474Z INFO: Gathering OVN_Southbound from ovnkube-node-szlgt...
2026-09-11T02:57:24.457543509Z INFO: Gathering OVN_Southbound from ovnkube-node-xjmbk...
2026-09-11T02:57:24.524199778Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T02:57:24.530830178Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:24.603927878Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-11T02:57:24.771861447Z tar: Removing leading `/' from member names
2026-09-11T02:57:24.782729054Z tar: Removing leading `/' from member names
2026-09-11T02:57:24.796462289Z tar: Removing leading `/' from member names
2026-09-11T02:57:24.808257172Z tar: Removing leading `/' from member names
2026-09-11T02:57:24.842785422Z Wrote inspect data to must-gather.
2026-09-11T02:57:24.843971066Z tar: Removing leading `/' from member names
2026-09-11T02:57:24.846833025Z tar: Removing leading `/' from member names
2026-09-11T02:57:24.878417509Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:24.880700889Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-11T02:57:24.959286213Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-11T02:57:25.084449251Z Wrote inspect data to must-gather.
2026-09-11T02:57:25.128330213Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:25.229296270Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-11T02:57:25.278843903Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T02:57:25.317686030Z INFO: Waiting for network log collection to complete ...
2026-09-11T02:57:25.317783593Z INFO: Waiting for ovnk database copies to complete ...
2026-09-11T02:57:25.318286404Z INFO: Copying ovnk databases complete.
2026-09-11T02:57:25.320268251Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-09-11T02:57:25.323225164Z ovnk_database_store/
2026-09-11T02:57:25.323225164Z ovnk_database_store/ovnkube-node-jv2sz_sbdb
2026-09-11T02:57:25.344806231Z ovnk_database_store/ovnkube-node-jv2sz_nbdb
2026-09-11T02:57:25.350629886Z ovnk_database_store/ovnkube-node-xjmbk_nbdb
2026-09-11T02:57:25.357728042Z ovnk_database_store/ovnkube-node-xjmbk_sbdb
2026-09-11T02:57:25.377171995Z ovnk_database_store/ovnkube-node-szlgt_nbdb
2026-09-11T02:57:25.378045556Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:25.380426626Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-11T02:57:25.382567807Z ovnk_database_store/ovnkube-node-szlgt_sbdb
2026-09-11T02:57:25.402092284Z INFO: Network log collection complete.
2026-09-11T02:57:25.426610828Z Wrote inspect data to must-gather.
2026-09-11T02:57:25.475592529Z Gathering data for ns/openshift-network-console...
2026-09-11T02:57:25.587869088Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:25.644920885Z INFO: Getting status from alertmanager-main-0
2026-09-11T02:57:25.724299125Z Wrote inspect data to must-gather.
2026-09-11T02:57:25.814962615Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:25.817867096Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-11T02:57:25.941242858Z Gathering data for ns/openshift-console-operator...
2026-09-11T02:57:26.042203581Z Wrote inspect data to must-gather.
2026-09-11T02:57:26.060326723Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:26.266510592Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:26.269602933Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-11T02:57:26.308693895Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-11T02:57:26.311083469Z INFO: Fallback to identify the container image from release info
2026-09-11T02:57:26.324470769Z Gathering data for ns/openshift-console...
2026-09-11T02:57:26.434686294Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:26.616161441Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:57:26.859126328Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-11T02:57:27.284445469Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-11T02:57:27.303987546Z Done executing Istio gather script
2026-09-11T02:57:27.350810028Z Gathering data for ns/openshift-dns-operator...
2026-09-11T02:57:27.422794394Z daemonset.apps/perf-node-gather-daemonset created
2026-09-11T02:57:27.537780298Z Waiting for performance profile collector pods to become ready: 1
2026-09-11T02:57:27.626058957Z Gathering data for ns/openshift-dns...
2026-09-11T02:57:28.175413721Z Gathering data for ns/openshift-image-registry...
2026-09-11T02:57:28.193059728Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T02:57:28.593797675Z Gathering data for ns/openshift-ingress-operator...
2026-09-11T02:57:28.663188685Z Waiting for performance profile collector pods to become ready: 2
2026-09-11T02:57:28.897591192Z Gathering data for ns/openshift-ingress...
2026-09-11T02:57:29.216456553Z Gathering data for ns/openshift-ingress-canary...
2026-09-11T02:57:29.554559408Z Gathering data for ns/openshift-insights...
2026-09-11T02:57:29.766160033Z Waiting for performance profile collector pods to become ready: 3
2026-09-11T02:57:30.890169586Z Waiting for performance profile collector pods to become ready: 4
2026-09-11T02:57:31.337033766Z Gathering data for ns/openshift-config...
2026-09-11T02:57:31.618946617Z Gathering data for ns/openshift-config-managed...
2026-09-11T02:57:32.015258013Z Waiting for performance profile collector pods to become ready: 5
2026-09-11T02:57:32.018494731Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-11T02:57:32.258983495Z Gathering data for ns/openshift-kube-apiserver...
2026-09-11T02:57:32.493544530Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-11T02:57:32.743280657Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-11T02:57:33.002955490Z Gathering data for ns/openshift-kube-scheduler...
2026-09-11T02:57:33.133181375Z Waiting for performance profile collector pods to become ready: 6
2026-09-11T02:57:33.198623714Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T02:57:33.236785955Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-11T02:57:33.490066271Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-11T02:57:33.782290878Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-11T02:57:34.051536638Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-11T02:57:34.252753828Z Waiting for performance profile collector pods to become ready: 7
2026-09-11T02:57:34.490422748Z Gathering data for ns/openshift-multus...
2026-09-11T02:57:35.358587507Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-11T02:57:35.609285720Z Collecting performance related data for node ip-10-0-138-210.ec2.internal
2026-09-11T02:57:35.619129344Z Collecting performance related data for node ip-10-0-132-61.ec2.internal
2026-09-11T02:57:35.626754228Z Collecting performance related data for node ip-10-0-132-88.ec2.internal
2026-09-11T02:57:35.882137414Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-11T02:57:37.165911744Z Gathering data for ns/openshift-host-network...
2026-09-11T02:57:37.431422815Z Gathering data for ns/openshift-network-diagnostics...
2026-09-11T02:57:37.948777619Z Gathering data for ns/openshift-network-node-identity...
2026-09-11T02:57:38.204742912Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T02:57:38.402626235Z Gathering data for ns/openshift-network-operator...
2026-09-11T02:57:38.763911061Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-11T02:57:39.006281579Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-11T02:57:39.462272240Z Gathering data for ns/openshift-apiserver-operator...
2026-09-11T02:57:39.742306360Z Gathering data for ns/openshift-apiserver...
2026-09-11T02:57:40.147200467Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-11T02:57:40.430177615Z Gathering data for ns/openshift-controller-manager...
2026-09-11T02:57:40.685183334Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-11T02:57:41.310103467Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-11T02:57:41.560805922Z Gathering data for ns/openshift-service-ca-operator...
2026-09-11T02:57:41.844034388Z Gathering data for ns/openshift-service-ca...
2026-09-11T02:57:42.172940724Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-11T02:57:43.169213764Z Wrote inspect data to must-gather.
2026-09-11T02:57:43.169262910Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T02:57:43.169262910Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-11T02:57:43.209901728Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T02:57:43.508151034Z Collecting kubelet logs for node ip-10-0-138-210.ec2.internal
2026-09-11T02:57:43.510085207Z Collecting kubelet logs for node ip-10-0-132-61.ec2.internal
2026-09-11T02:57:43.511885529Z Collecting kubelet logs for node ip-10-0-132-88.ec2.internal
2026-09-11T02:57:43.801848556Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vx2l8 namespace
2026-09-11T02:57:43.807149538Z INFO: Node performance data collection complete.
2026-09-11T02:57:43.934774124Z error: the server doesn't have a resource type "clusters"
2026-09-11T02:57:44.090794142Z Caches written to disk