2026-07-27T12:32:59.884357974Z [disk usage checker] Started
2026-07-27T12:32:59.887843275Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T12:33:00.092681888Z Gathering data for ns/openshift-cluster-version...
2026-07-27T12:33:00.227982898Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-27T12:33:00.318030938Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T12:33:00.407908111Z Gathering data for ns/default...
2026-07-27T12:33:00.712088897Z Gathering data for ns/openshift...
2026-07-27T12:33:01.143159682Z Gathering data for ns/kube-system...
2026-07-27T12:33:01.862833049Z Gathering data for ns/openshift-etcd...
2026-07-27T12:33:02.176673546Z Wrote inspect data to must-gather.
2026-07-27T12:33:02.176715889Z error: inspection completed with the errors occurred while gathering data:
2026-07-27T12:33:02.176715889Z     namespaces "assisted-installer" not found
2026-07-27T12:33:02.640521232Z Waiting on subprocesses to finish execution.
2026-07-27T12:33:02.646543684Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-27T12:33:02.649106624Z INFO: Gathering HAProxy config files
2026-07-27T12:33:02.652744680Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-27T12:33:02.652744680Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-27T12:33:02.656457730Z Executing Istio gather script
2026-07-27T12:33:02.702155151Z INFO: Waiting for node performance related collection to complete ...
2026-07-27T12:33:02.721846989Z INFO: Collecting host service logs for crio
2026-07-27T12:33:02.722412030Z INFO: Collecting host service logs for kubelet
2026-07-27T12:33:02.722828045Z INFO: Collecting host service logs for rpm-ostreed
2026-07-27T12:33:02.723216442Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-27T12:33:02.723581602Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-27T12:33:02.723917763Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-27T12:33:02.724283685Z INFO: Collecting host service logs for NetworkManager
2026-07-27T12:33:02.724644386Z INFO: Collecting host service logs for openvswitch
2026-07-27T12:33:02.724976342Z INFO: Collecting host service logs for ovs-configuration
2026-07-27T12:33:02.725441165Z INFO: Collecting host service logs for ovsdb-server
2026-07-27T12:33:02.725848277Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-27T12:33:02.726298568Z INFO: Waiting for worker host service log collection to complete ...
2026-07-27T12:33:03.060229086Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-27T12:33:03.060276552Z INFO: on-disk MachineConfig config collection complete.
2026-07-27T12:33:03.195865252Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-zjvhm
2026-07-27T12:33:03.195918515Z insights-runtime-extractor-752jh
2026-07-27T12:33:03.195938952Z insights-runtime-extractor-mdfcs
2026-07-27T12:33:03.195958993Z insights-runtime-extractor-pfvcz
2026-07-27T12:33:03.453269631Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-27T12:33:03.553876262Z No resources found in openshift-etcd namespace.
2026-07-27T12:33:03.615559783Z No resources found
2026-07-27T12:33:03.623301352Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-27T12:33:03.676119538Z INFO: "metallb-operator" not detected. Skipping.
2026-07-27T12:33:03.898112851Z Wrote inspect data to must-gather.
2026-07-27T12:33:03.904289133Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-27T12:33:04.322261294Z error: the server doesn't have a resource type "performanceprofile"
2026-07-27T12:33:04.335501071Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-27T12:33:04.364427221Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-27T12:33:04.364427221Z See 'oc rsync -h' for help and examples
2026-07-27T12:33:04.437286939Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-27T12:33:04.547113570Z Wrote inspect data to must-gather.
2026-07-27T12:33:04.638300817Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T12:33:04.755748909Z ERROR: No running kube-apiserver pods found
2026-07-27T12:33:04.813886724Z INFO: Worker host service log collection to complete.
2026-07-27T12:33:04.880679111Z No resources found
2026-07-27T12:33:04.955579272Z Wrote inspect data to must-gather.
2026-07-27T12:33:04.972669570Z Wrote inspect data to must-gather.
2026-07-27T12:33:04.995193563Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-27T12:33:05.015459307Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T12:33:05.034552271Z Gathering data for ns/openshift-monitoring...
2026-07-27T12:33:05.137394753Z Wrote inspect data to must-gather.
2026-07-27T12:33:05.169859891Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-27T12:33:05.247307088Z Wrote inspect data to must-gather.
2026-07-27T12:33:05.284731123Z INFO: 'previous-logs' folder not found on ip-10-0-131-159.ec2.internal, skipping...
2026-07-27T12:33:05.296811268Z Wrote inspect data to must-gather.
2026-07-27T12:33:05.437930635Z error: the server doesn't have a resource type "machineconfigs"
2026-07-27T12:33:05.529211400Z tar: Removing leading `/' from member names
2026-07-27T12:33:05.533455737Z INFO: HAProxy config collection complete.
2026-07-27T12:33:05.542483237Z Wrote inspect data to must-gather.
2026-07-27T12:33:05.640378362Z INFO: 'previous-logs' folder not found on ip-10-0-136-26.ec2.internal, skipping...
2026-07-27T12:33:05.659924659Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-27T12:33:05.692347377Z INFO: Getting rules from prometheus-k8s-0
2026-07-27T12:33:05.889326954Z Wrote inspect data to must-gather.
2026-07-27T12:33:05.943664888Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-27T12:33:05.959415031Z Wrote inspect data to must-gather.
2026-07-27T12:33:05.968837292Z INFO: 'previous-logs' folder not found on ip-10-0-141-3.ec2.internal, skipping...
2026-07-27T12:33:05.969464845Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-27T12:33:05.969571328Z INFO: Machine Config Daemon termination log collection complete.
2026-07-27T12:33:06.134690016Z INFO: Getting status/config from prometheus-k8s-0
2026-07-27T12:33:06.147012708Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-27T12:33:06.171014223Z Wrote inspect data to must-gather.
2026-07-27T12:33:06.355468014Z Wrote inspect data to must-gather.
2026-07-27T12:33:06.457687028Z INFO: INTERCONNECT MODE
2026-07-27T12:33:06.457794273Z INFO: Gathering ovn-kubernetes DBs
2026-07-27T12:33:06.534112325Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-27T12:33:06.597025168Z Wrote inspect data to must-gather.
2026-07-27T12:33:06.636724209Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-27T12:33:06.654435649Z INFO: Gathering OVN_Northbound from ovnkube-node-6w2hz...
2026-07-27T12:33:06.654760635Z INFO: Gathering OVN_Northbound from ovnkube-node-ktgz9...
2026-07-27T12:33:06.655088629Z INFO: Gathering OVN_Northbound from ovnkube-node-p6q4f...
2026-07-27T12:33:06.655406007Z INFO: Gathering OVN_Southbound from ovnkube-node-6w2hz...
2026-07-27T12:33:06.655984029Z INFO: Gathering OVN_Southbound from ovnkube-node-ktgz9...
2026-07-27T12:33:06.656308605Z INFO: Gathering OVN_Southbound from ovnkube-node-p6q4f...
2026-07-27T12:33:06.785316499Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-27T12:33:06.910895181Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-27T12:33:06.979288279Z tar: Removing leading `/' from member names
2026-07-27T12:33:06.986504420Z Wrote inspect data to must-gather.
2026-07-27T12:33:07.001687349Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:07.013881883Z tar: Removing leading `/' from member names
2026-07-27T12:33:07.022429844Z tar: Removing leading `/' from member names
2026-07-27T12:33:07.048691570Z tar: Removing leading `/' from member names
2026-07-27T12:33:07.049078787Z tar: Removing leading `/' from member names
2026-07-27T12:33:07.080580312Z tar: Removing leading `/' from member names
2026-07-27T12:33:07.210179769Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-27T12:33:07.253095151Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:07.257235187Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-27T12:33:07.262765116Z Wrote inspect data to must-gather.
2026-07-27T12:33:07.505729930Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-27T12:33:07.517633360Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:07.524401003Z W0727 12:33:07.524338    1453 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-27T12:33:07.536041085Z Wrote inspect data to must-gather.
2026-07-27T12:33:07.597566579Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T12:33:07.603292601Z INFO: Waiting for network log collection to complete ...
2026-07-27T12:33:07.603336517Z INFO: Waiting for ovnk database copies to complete ...
2026-07-27T12:33:07.604232623Z INFO: Copying ovnk databases complete.
2026-07-27T12:33:07.606486417Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-27T12:33:07.610530335Z ovnk_database_store/
2026-07-27T12:33:07.610603213Z ovnk_database_store/ovnkube-node-6w2hz_nbdb
2026-07-27T12:33:07.619568719Z ovnk_database_store/ovnkube-node-p6q4f_sbdb
2026-07-27T12:33:07.640406170Z ovnk_database_store/ovnkube-node-ktgz9_nbdb
2026-07-27T12:33:07.648234018Z ovnk_database_store/ovnkube-node-p6q4f_nbdb
2026-07-27T12:33:07.657972263Z ovnk_database_store/ovnkube-node-6w2hz_sbdb
2026-07-27T12:33:07.677204188Z ovnk_database_store/ovnkube-node-ktgz9_sbdb
2026-07-27T12:33:07.700769829Z INFO: Network log collection complete.
2026-07-27T12:33:07.766412629Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:07.769651910Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-27T12:33:07.804998502Z Gathering data for ns/openshift-network-console...
2026-07-27T12:33:07.879629716Z Wrote inspect data to must-gather.
2026-07-27T12:33:07.942153611Z INFO: Getting status from alertmanager-main-0
2026-07-27T12:33:07.986330229Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:08.215541718Z Wrote inspect data to must-gather.
2026-07-27T12:33:08.226258902Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:08.229152709Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-27T12:33:08.285200004Z Gathering data for ns/openshift-console-operator...
2026-07-27T12:33:08.470809103Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:08.502198159Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-27T12:33:08.504987312Z INFO: Fallback to identify the container image from release info
2026-07-27T12:33:08.687449268Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:08.690188938Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-27T12:33:08.692400307Z Gathering data for ns/openshift-console...
2026-07-27T12:33:08.857938304Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:09.063578237Z Wrote inspect data to /must-gather/istio.
2026-07-27T12:33:09.120310194Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-27T12:33:09.238935181Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-27T12:33:09.271218705Z daemonset.apps/perf-node-gather-daemonset created
2026-07-27T12:33:09.408826469Z Waiting for performance profile collector pods to become ready: 1
2026-07-27T12:33:09.832840539Z Done executing Istio gather script
2026-07-27T12:33:09.852299130Z Gathering data for ns/openshift-dns-operator...
2026-07-27T12:33:10.029439244Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T12:33:10.156327933Z Gathering data for ns/openshift-dns...
2026-07-27T12:33:10.538070603Z Waiting for performance profile collector pods to become ready: 2
2026-07-27T12:33:10.779974382Z Gathering data for ns/openshift-image-registry...
2026-07-27T12:33:11.234413812Z Gathering data for ns/openshift-ingress-operator...
2026-07-27T12:33:11.556796506Z Gathering data for ns/openshift-ingress...
2026-07-27T12:33:11.667418928Z Waiting for performance profile collector pods to become ready: 3
2026-07-27T12:33:11.906240360Z Gathering data for ns/openshift-ingress-canary...
2026-07-27T12:33:12.289382971Z Gathering data for ns/openshift-insights...
2026-07-27T12:33:12.794629452Z Waiting for performance profile collector pods to become ready: 4
2026-07-27T12:33:13.920298221Z Waiting for performance profile collector pods to become ready: 5
2026-07-27T12:33:14.206805222Z Gathering data for ns/openshift-jobset-operator...
2026-07-27T12:33:15.021629677Z Gathering data for ns/openshift-config...
2026-07-27T12:33:15.035315826Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T12:33:15.036900853Z Waiting for performance profile collector pods to become ready: 6
2026-07-27T12:33:15.283385222Z Gathering data for ns/openshift-config-managed...
2026-07-27T12:33:15.700103996Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-27T12:33:15.964565218Z Gathering data for ns/openshift-kube-apiserver...
2026-07-27T12:33:16.153964090Z Waiting for performance profile collector pods to become ready: 7
2026-07-27T12:33:16.235717847Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-27T12:33:16.524979290Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-27T12:33:16.797290646Z Gathering data for ns/openshift-kube-scheduler...
2026-07-27T12:33:17.050683483Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-27T12:33:17.285191159Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-27T12:33:17.358816659Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-27T12:33:17.552090405Z Collecting performance related data for node ip-10-0-141-3.ec2.internal
2026-07-27T12:33:17.559531378Z Collecting performance related data for node ip-10-0-131-159.ec2.internal
2026-07-27T12:33:17.569343801Z Collecting performance related data for node ip-10-0-136-26.ec2.internal
2026-07-27T12:33:17.742819322Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-27T12:33:18.100492785Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-27T12:33:18.726723189Z Gathering data for ns/openshift-multus...
2026-07-27T12:33:20.040472943Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T12:33:20.255200164Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-27T12:33:21.510610859Z Gathering data for ns/openshift-host-network...
2026-07-27T12:33:21.829144189Z Gathering data for ns/openshift-network-diagnostics...
2026-07-27T12:33:22.381079268Z Gathering data for ns/openshift-network-node-identity...
2026-07-27T12:33:22.808698979Z Gathering data for ns/openshift-network-operator...
2026-07-27T12:33:23.203345823Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-27T12:33:23.521640513Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-27T12:33:24.002345428Z Gathering data for ns/openshift-apiserver-operator...
2026-07-27T12:33:24.291186151Z Gathering data for ns/openshift-apiserver...
2026-07-27T12:33:24.696955769Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-27T12:33:24.976225599Z Gathering data for ns/openshift-controller-manager...
2026-07-27T12:33:25.045731428Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T12:33:25.245290622Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-27T12:33:25.899545209Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-27T12:33:26.144977261Z Collecting kubelet logs for node ip-10-0-141-3.ec2.internal
2026-07-27T12:33:26.146909890Z Collecting kubelet logs for node ip-10-0-131-159.ec2.internal
2026-07-27T12:33:26.149401338Z Collecting kubelet logs for node ip-10-0-136-26.ec2.internal
2026-07-27T12:33:26.355600173Z Gathering data for ns/openshift-service-ca-operator...
2026-07-27T12:33:26.490221340Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-8d887 namespace
2026-07-27T12:33:26.501727806Z INFO: Node performance data collection complete.
2026-07-27T12:33:26.759469644Z Gathering data for ns/openshift-service-ca...
2026-07-27T12:33:27.140609170Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-27T12:33:28.196025142Z Wrote inspect data to must-gather.
2026-07-27T12:33:28.196198125Z error: inspection completed with the errors occurred while gathering data:
2026-07-27T12:33:28.196198125Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-27T12:33:28.320980698Z error: the server doesn't have a resource type "clusters"
2026-07-27T12:33:28.441760947Z Caches written to disk