2026-07-20T19:32:05.320668757Z [disk usage checker] Started
2026-07-20T19:32:05.323771042Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:32:05.508130100Z Gathering data for ns/openshift-cluster-version...
2026-07-20T19:32:05.627384286Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T19:32:05.710477798Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:32:05.786760695Z Gathering data for ns/default...
2026-07-20T19:32:06.057416639Z Gathering data for ns/openshift...
2026-07-20T19:32:06.454890855Z Gathering data for ns/kube-system...
2026-07-20T19:32:07.122340547Z Gathering data for ns/openshift-etcd...
2026-07-20T19:32:07.404036028Z Wrote inspect data to must-gather.
2026-07-20T19:32:07.404120676Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T19:32:07.404120676Z     namespaces "assisted-installer" not found
2026-07-20T19:32:07.773114460Z Waiting on subprocesses to finish execution.
2026-07-20T19:32:07.778606038Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-20T19:32:07.783012474Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-20T19:32:07.783485721Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-20T19:32:07.787415460Z INFO: Collecting host service logs for crio
2026-07-20T19:32:07.787724122Z INFO: Collecting host service logs for kubelet
2026-07-20T19:32:07.788070581Z INFO: Collecting host service logs for rpm-ostreed
2026-07-20T19:32:07.788407014Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-20T19:32:07.788767736Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-20T19:32:07.788958563Z INFO: Gathering HAProxy config files
2026-07-20T19:32:07.789142011Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-20T19:32:07.789469632Z INFO: Collecting host service logs for NetworkManager
2026-07-20T19:32:07.789803017Z INFO: Collecting host service logs for openvswitch
2026-07-20T19:32:07.790131729Z INFO: Collecting host service logs for ovs-configuration
2026-07-20T19:32:07.790462688Z INFO: Collecting host service logs for ovsdb-server
2026-07-20T19:32:07.790835652Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-20T19:32:07.791217302Z INFO: Waiting for worker host service log collection to complete ...
2026-07-20T19:32:07.859032624Z INFO: Waiting for node performance related collection to complete ...
2026-07-20T19:32:07.913884310Z Executing Istio gather script
2026-07-20T19:32:08.376077394Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-20T19:32:08.517921276Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-20T19:32:08.640831586Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-mq24w
2026-07-20T19:32:08.640922845Z insights-runtime-extractor-6lmhs
2026-07-20T19:32:08.640954519Z insights-runtime-extractor-cn6d2
2026-07-20T19:32:08.642132764Z insights-runtime-extractor-ctcp5
2026-07-20T19:32:08.763221725Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-20T19:32:08.763259919Z INFO: on-disk MachineConfig config collection complete.
2026-07-20T19:32:08.871359010Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-20T19:32:08.908165945Z No resources found in openshift-etcd namespace.
2026-07-20T19:32:09.074003979Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-20T19:32:09.124240368Z No resources found
2026-07-20T19:32:09.333756412Z INFO: Worker host service log collection to complete.
2026-07-20T19:32:09.380401778Z error: the server doesn't have a resource type "performanceprofile"
2026-07-20T19:32:09.466397479Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:32:09.505407329Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-20T19:32:09.505407329Z See 'oc rsync -h' for help and examples
2026-07-20T19:32:09.668249153Z INFO: "metallb-operator" not detected. Skipping.
2026-07-20T19:32:09.758291469Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-20T19:32:09.786634171Z Wrote inspect data to must-gather.
2026-07-20T19:32:09.811250765Z Wrote inspect data to must-gather.
2026-07-20T19:32:09.917807412Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-20T19:32:09.946639262Z Wrote inspect data to must-gather.
2026-07-20T19:32:09.948086024Z No resources found
2026-07-20T19:32:09.994397444Z Wrote inspect data to must-gather.
2026-07-20T19:32:10.060894146Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-20T19:32:10.076638118Z ERROR: No running kube-apiserver pods found
2026-07-20T19:32:10.148781975Z Wrote inspect data to must-gather.
2026-07-20T19:32:10.169992144Z Wrote inspect data to must-gather.
2026-07-20T19:32:10.189821857Z Wrote inspect data to must-gather.
2026-07-20T19:32:10.298124760Z INFO: 'previous-logs' folder not found on ip-10-0-128-75.ec2.internal, skipping...
2026-07-20T19:32:10.331343895Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:32:10.378989398Z tar: Removing leading `/' from member names
2026-07-20T19:32:10.384817662Z INFO: HAProxy config collection complete.
2026-07-20T19:32:10.429963989Z error: the server doesn't have a resource type "machineconfigs"
2026-07-20T19:32:10.449319579Z Gathering data for ns/openshift-monitoring...
2026-07-20T19:32:10.480355674Z Wrote inspect data to must-gather.
2026-07-20T19:32:10.516275585Z INFO: Getting rules from prometheus-k8s-0
2026-07-20T19:32:10.614206278Z INFO: 'previous-logs' folder not found on ip-10-0-132-253.ec2.internal, skipping...
2026-07-20T19:32:10.643635908Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-20T19:32:10.678276174Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-20T19:32:10.778500562Z Wrote inspect data to must-gather.
2026-07-20T19:32:10.870721145Z Wrote inspect data to must-gather.
2026-07-20T19:32:10.893296451Z INFO: 'previous-logs' folder not found on ip-10-0-138-147.ec2.internal, skipping...
2026-07-20T19:32:10.893379266Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-20T19:32:10.893416912Z INFO: Machine Config Daemon termination log collection complete.
2026-07-20T19:32:10.984588656Z INFO: Getting status/config from prometheus-k8s-0
2026-07-20T19:32:11.023931228Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-20T19:32:11.034181240Z Wrote inspect data to must-gather.
2026-07-20T19:32:11.228043542Z Wrote inspect data to must-gather.
2026-07-20T19:32:11.341958018Z INFO: INTERCONNECT MODE
2026-07-20T19:32:11.341994728Z INFO: Gathering ovn-kubernetes DBs
2026-07-20T19:32:11.393161822Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-20T19:32:11.454025906Z Wrote inspect data to must-gather.
2026-07-20T19:32:11.504481043Z INFO: Gathering OVN_Northbound from ovnkube-node-557ld...
2026-07-20T19:32:11.504718952Z INFO: Gathering OVN_Northbound from ovnkube-node-jb8wn...
2026-07-20T19:32:11.504968118Z INFO: Gathering OVN_Northbound from ovnkube-node-r82zg...
2026-07-20T19:32:11.505237753Z INFO: Gathering OVN_Southbound from ovnkube-node-557ld...
2026-07-20T19:32:11.505469007Z INFO: Gathering OVN_Southbound from ovnkube-node-jb8wn...
2026-07-20T19:32:11.505785128Z INFO: Gathering OVN_Southbound from ovnkube-node-r82zg...
2026-07-20T19:32:11.653315835Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-20T19:32:11.734217499Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-20T19:32:11.748296728Z Wrote inspect data to must-gather.
2026-07-20T19:32:11.806812429Z tar: Removing leading `/' from member names
2026-07-20T19:32:11.809675802Z tar: Removing leading `/' from member names
2026-07-20T19:32:11.815206158Z tar: Removing leading `/' from member names
2026-07-20T19:32:11.832255554Z tar: Removing leading `/' from member names
2026-07-20T19:32:11.848936415Z tar: Removing leading `/' from member names
2026-07-20T19:32:11.870685708Z tar: Removing leading `/' from member names
2026-07-20T19:32:11.985076916Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:11.985373917Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-20T19:32:12.030962193Z Wrote inspect data to must-gather.
2026-07-20T19:32:12.046079520Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T19:32:12.222304178Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:12.225212164Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-20T19:32:12.244103902Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-20T19:32:12.286165141Z Wrote inspect data to must-gather.
2026-07-20T19:32:12.375373054Z INFO: Waiting for network log collection to complete ...
2026-07-20T19:32:12.375420705Z INFO: Waiting for ovnk database copies to complete ...
2026-07-20T19:32:12.375934619Z INFO: Copying ovnk databases complete.
2026-07-20T19:32:12.378060447Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-20T19:32:12.380346006Z ovnk_database_store/
2026-07-20T19:32:12.380346006Z ovnk_database_store/ovnkube-node-557ld_nbdb
2026-07-20T19:32:12.388950967Z ovnk_database_store/ovnkube-node-jb8wn_nbdb
2026-07-20T19:32:12.394179098Z ovnk_database_store/ovnkube-node-r82zg_nbdb
2026-07-20T19:32:12.401399732Z ovnk_database_store/ovnkube-node-557ld_sbdb
2026-07-20T19:32:12.418339495Z ovnk_database_store/ovnkube-node-jb8wn_sbdb
2026-07-20T19:32:12.436383823Z ovnk_database_store/ovnkube-node-r82zg_sbdb
2026-07-20T19:32:12.450669122Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:12.456015853Z INFO: Network log collection complete.
2026-07-20T19:32:12.500795894Z Wrote inspect data to must-gather.
2026-07-20T19:32:12.650794447Z INFO: Getting status from alertmanager-main-0
2026-07-20T19:32:12.675742720Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:12.678380829Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-20T19:32:12.683478183Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:32:12.799942545Z Wrote inspect data to must-gather.
2026-07-20T19:32:12.872723914Z Gathering data for ns/openshift-network-console...
2026-07-20T19:32:12.881320491Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:13.059756643Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-20T19:32:13.061946861Z INFO: Fallback to identify the container image from release info
2026-07-20T19:32:13.074894198Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:13.077684606Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-20T19:32:13.279530263Z Gathering data for ns/openshift-console-operator...
2026-07-20T19:32:13.288975696Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:13.467860239Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:13.470197807Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-20T19:32:13.630025892Z Gathering data for ns/openshift-console...
2026-07-20T19:32:13.633330465Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:13.758407978Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-20T19:32:13.832930601Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:32:13.874738449Z daemonset.apps/perf-node-gather-daemonset created
2026-07-20T19:32:14.021392202Z Waiting for performance profile collector pods to become ready: 1
2026-07-20T19:32:14.158384575Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-20T19:32:14.531094391Z Done executing Istio gather script
2026-07-20T19:32:14.643905007Z Gathering data for ns/openshift-dns-operator...
2026-07-20T19:32:14.882964670Z Gathering data for ns/openshift-dns...
2026-07-20T19:32:15.130110651Z Waiting for performance profile collector pods to become ready: 2
2026-07-20T19:32:15.336377988Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:32:15.422983339Z Gathering data for ns/openshift-image-registry...
2026-07-20T19:32:15.855352632Z Gathering data for ns/openshift-ingress-operator...
2026-07-20T19:32:16.180277563Z Gathering data for ns/openshift-ingress...
2026-07-20T19:32:16.246983796Z Waiting for performance profile collector pods to become ready: 3
2026-07-20T19:32:16.490330907Z Gathering data for ns/openshift-ingress-canary...
2026-07-20T19:32:16.836596901Z Gathering data for ns/openshift-insights...
2026-07-20T19:32:17.366234879Z Waiting for performance profile collector pods to become ready: 4
2026-07-20T19:32:18.470314201Z Waiting for performance profile collector pods to become ready: 5
2026-07-20T19:32:18.515415287Z Gathering data for ns/openshift-jobset-operator...
2026-07-20T19:32:19.172045395Z Gathering data for ns/openshift-config...
2026-07-20T19:32:19.405077396Z Gathering data for ns/openshift-config-managed...
2026-07-20T19:32:19.575330857Z Waiting for performance profile collector pods to become ready: 6
2026-07-20T19:32:19.802733500Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-20T19:32:20.087316638Z Gathering data for ns/openshift-kube-apiserver...
2026-07-20T19:32:20.317246435Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-20T19:32:20.342584805Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:32:20.582805669Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-20T19:32:20.696426645Z Waiting for performance profile collector pods to become ready: 7
2026-07-20T19:32:20.890015895Z Gathering data for ns/openshift-kube-scheduler...
2026-07-20T19:32:21.114153307Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-20T19:32:21.378940899Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-20T19:32:21.664639542Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-20T19:32:21.814346702Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-20T19:32:21.961763052Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-20T19:32:22.058764928Z Collecting performance related data for node ip-10-0-128-75.ec2.internal
2026-07-20T19:32:22.065002019Z Collecting performance related data for node ip-10-0-132-253.ec2.internal
2026-07-20T19:32:22.071648959Z Collecting performance related data for node ip-10-0-138-147.ec2.internal
2026-07-20T19:32:22.606969506Z Gathering data for ns/openshift-multus...
2026-07-20T19:32:24.006051497Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-20T19:32:25.180231493Z Gathering data for ns/openshift-host-network...
2026-07-20T19:32:25.349738013Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:32:25.450830121Z Gathering data for ns/openshift-network-diagnostics...
2026-07-20T19:32:25.960723313Z Gathering data for ns/openshift-network-node-identity...
2026-07-20T19:32:26.340267484Z Gathering data for ns/openshift-network-operator...
2026-07-20T19:32:26.706782718Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-20T19:32:26.940935897Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-20T19:32:27.394126399Z Gathering data for ns/openshift-apiserver-operator...
2026-07-20T19:32:27.665823730Z Gathering data for ns/openshift-apiserver...
2026-07-20T19:32:28.021456029Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-20T19:32:28.293806376Z Gathering data for ns/openshift-controller-manager...
2026-07-20T19:32:28.536377654Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-20T19:32:29.159545423Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-20T19:32:29.424443001Z Gathering data for ns/openshift-service-ca-operator...
2026-07-20T19:32:29.719457327Z Gathering data for ns/openshift-service-ca...
2026-07-20T19:32:29.831996601Z Collecting kubelet logs for node ip-10-0-128-75.ec2.internal
2026-07-20T19:32:29.833759090Z Collecting kubelet logs for node ip-10-0-132-253.ec2.internal
2026-07-20T19:32:29.835385623Z Collecting kubelet logs for node ip-10-0-138-147.ec2.internal
2026-07-20T19:32:30.096157167Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-20T19:32:30.156357556Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-8zlfh namespace
2026-07-20T19:32:30.161005414Z INFO: Node performance data collection complete.
2026-07-20T19:32:30.358754547Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:32:30.955210254Z Wrote inspect data to must-gather.
2026-07-20T19:32:30.955248342Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T19:32:30.955248342Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-20T19:32:31.073247318Z error: the server doesn't have a resource type "clusters"
2026-07-20T19:32:31.209043556Z Caches written to disk