2026-07-21T18:15:50.658387774Z [disk usage checker] Started
2026-07-21T18:15:50.661788832Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T18:15:50.863259570Z Gathering data for ns/openshift-cluster-version...
2026-07-21T18:15:51.008626802Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-21T18:15:51.104070449Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T18:15:51.188570700Z Gathering data for ns/default...
2026-07-21T18:15:51.531327752Z Gathering data for ns/openshift...
2026-07-21T18:15:51.978374430Z Gathering data for ns/kube-system...
2026-07-21T18:15:52.692878305Z Gathering data for ns/openshift-etcd...
2026-07-21T18:15:53.039207537Z Wrote inspect data to must-gather.
2026-07-21T18:15:53.039298802Z error: inspection completed with the errors occurred while gathering data:
2026-07-21T18:15:53.039298802Z     namespaces "assisted-installer" not found
2026-07-21T18:15:53.470854181Z Waiting on subprocesses to finish execution.
2026-07-21T18:15:53.476862746Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-21T18:15:53.485328657Z Executing Istio gather script
2026-07-21T18:15:53.487197148Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-21T18:15:53.490400583Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-21T18:15:53.497289866Z INFO: Gathering HAProxy config files
2026-07-21T18:15:53.515638695Z INFO: Collecting host service logs for crio
2026-07-21T18:15:53.515963827Z INFO: Collecting host service logs for kubelet
2026-07-21T18:15:53.516297558Z INFO: Collecting host service logs for rpm-ostreed
2026-07-21T18:15:53.516614552Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-21T18:15:53.516930638Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-21T18:15:53.517247079Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-21T18:15:53.517559947Z INFO: Collecting host service logs for NetworkManager
2026-07-21T18:15:53.517853460Z INFO: Collecting host service logs for openvswitch
2026-07-21T18:15:53.518178865Z INFO: Collecting host service logs for ovs-configuration
2026-07-21T18:15:53.518530642Z INFO: Collecting host service logs for ovsdb-server
2026-07-21T18:15:53.518874984Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-21T18:15:53.519321028Z INFO: Waiting for worker host service log collection to complete ...
2026-07-21T18:15:53.582534269Z INFO: Waiting for node performance related collection to complete ...
2026-07-21T18:15:54.091313663Z No resources found in openshift-etcd namespace.
2026-07-21T18:15:54.254575694Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-21T18:15:54.483603582Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-21T18:15:54.623837014Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-sl66l
2026-07-21T18:15:54.623887674Z insights-runtime-extractor-2jv9k
2026-07-21T18:15:54.623909396Z insights-runtime-extractor-dptd4
2026-07-21T18:15:54.623929946Z insights-runtime-extractor-rrwdh
2026-07-21T18:15:54.791252893Z INFO: "metallb-operator" not detected. Skipping.
2026-07-21T18:15:54.822817394Z No resources found
2026-07-21T18:15:54.859771626Z Wrote inspect data to must-gather.
2026-07-21T18:15:54.924960140Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-21T18:15:54.924960140Z See 'oc rsync -h' for help and examples
2026-07-21T18:15:54.946547815Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-21T18:15:54.946632476Z INFO: on-disk MachineConfig config collection complete.
2026-07-21T18:15:55.027285210Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-21T18:15:55.070732764Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-21T18:15:55.188431036Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-21T18:15:55.248996641Z INFO: Worker host service log collection to complete.
2026-07-21T18:15:55.285818276Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-21T18:15:55.359727651Z ERROR: No running kube-apiserver pods found
2026-07-21T18:15:55.506208315Z error: the server doesn't have a resource type "performanceprofile"
2026-07-21T18:15:55.600269986Z Gathering data for ns/openshift-monitoring...
2026-07-21T18:15:55.706077829Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T18:15:55.759856762Z Wrote inspect data to must-gather.
2026-07-21T18:15:55.788851912Z INFO: 'previous-logs' folder not found on ip-10-0-132-249.ec2.internal, skipping...
2026-07-21T18:15:56.029242449Z Wrote inspect data to must-gather.
2026-07-21T18:15:56.033876581Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-21T18:15:56.045989656Z Wrote inspect data to must-gather.
2026-07-21T18:15:56.119861754Z Wrote inspect data to must-gather.
2026-07-21T18:15:56.272048644Z tar: Removing leading `/' from member names
2026-07-21T18:15:56.280312051Z INFO: HAProxy config collection complete.
2026-07-21T18:15:56.347537207Z Wrote inspect data to must-gather.
2026-07-21T18:15:56.387409779Z Wrote inspect data to must-gather.
2026-07-21T18:15:56.401822267Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T18:15:56.420885875Z No resources found
2026-07-21T18:15:56.514899335Z INFO: Getting rules from prometheus-k8s-0
2026-07-21T18:15:56.545252444Z INFO: 'previous-logs' folder not found on ip-10-0-135-98.ec2.internal, skipping...
2026-07-21T18:15:56.608238989Z error: the server doesn't have a resource type "machineconfigs"
2026-07-21T18:15:56.713878282Z Wrote inspect data to must-gather.
2026-07-21T18:15:56.819656023Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-21T18:15:56.881455838Z INFO: 'previous-logs' folder not found on ip-10-0-136-21.ec2.internal, skipping...
2026-07-21T18:15:56.881508484Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-21T18:15:56.881518705Z INFO: Machine Config Daemon termination log collection complete.
2026-07-21T18:15:57.015577888Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-21T18:15:57.029557178Z INFO: Getting status/config from prometheus-k8s-0
2026-07-21T18:15:57.031279686Z Wrote inspect data to must-gather.
2026-07-21T18:15:57.089337841Z Wrote inspect data to must-gather.
2026-07-21T18:15:57.261485189Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-21T18:15:57.291855002Z Wrote inspect data to must-gather.
2026-07-21T18:15:57.507265831Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-21T18:15:57.508327877Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-21T18:15:57.547846075Z Wrote inspect data to must-gather.
2026-07-21T18:15:57.677488955Z INFO: INTERCONNECT MODE
2026-07-21T18:15:57.677588307Z INFO: Gathering ovn-kubernetes DBs
2026-07-21T18:15:57.798382384Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-21T18:15:57.832281251Z Wrote inspect data to must-gather.
2026-07-21T18:15:57.867885579Z INFO: Gathering OVN_Northbound from ovnkube-node-65gzx...
2026-07-21T18:15:57.868174274Z INFO: Gathering OVN_Northbound from ovnkube-node-dq7d6...
2026-07-21T18:15:57.868403126Z INFO: Gathering OVN_Northbound from ovnkube-node-v767z...
2026-07-21T18:15:57.868831571Z INFO: Gathering OVN_Southbound from ovnkube-node-65gzx...
2026-07-21T18:15:57.869160544Z INFO: Gathering OVN_Southbound from ovnkube-node-dq7d6...
2026-07-21T18:15:57.869442351Z INFO: Gathering OVN_Southbound from ovnkube-node-v767z...
2026-07-21T18:15:58.151345010Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-21T18:15:58.169262118Z tar: Removing leading `/' from member names
2026-07-21T18:15:58.257695670Z tar: Removing leading `/' from member names
2026-07-21T18:15:58.269027434Z tar: Removing leading `/' from member names
2026-07-21T18:15:58.273064668Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-21T18:15:58.274266469Z tar: Removing leading `/' from member names
2026-07-21T18:15:58.277528614Z tar: Removing leading `/' from member names
2026-07-21T18:15:58.282523029Z W0721 18:15:58.282480    1316 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-21T18:15:58.290057759Z Wrote inspect data to must-gather.
2026-07-21T18:15:58.306275767Z tar: Removing leading `/' from member names
2026-07-21T18:15:58.365686708Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T18:15:58.429356593Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:15:58.584852192Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-21T18:15:58.612456795Z Gathering data for ns/openshift-network-console...
2026-07-21T18:15:58.698868751Z Wrote inspect data to must-gather.
2026-07-21T18:15:58.715990108Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:15:58.720718559Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-21T18:15:58.874405018Z INFO: Waiting for network log collection to complete ...
2026-07-21T18:15:58.874536631Z INFO: Waiting for ovnk database copies to complete ...
2026-07-21T18:15:58.875033973Z INFO: Copying ovnk databases complete.
2026-07-21T18:15:58.877206497Z 3.8M	must-gather/network_logs/ovnk_database_store
2026-07-21T18:15:58.880111566Z ovnk_database_store/
2026-07-21T18:15:58.880111566Z ovnk_database_store/ovnkube-node-v767z_nbdb
2026-07-21T18:15:58.889803642Z ovnk_database_store/ovnkube-node-dq7d6_nbdb
2026-07-21T18:15:58.901313613Z ovnk_database_store/ovnkube-node-65gzx_sbdb
2026-07-21T18:15:58.926822286Z ovnk_database_store/ovnkube-node-65gzx_nbdb
2026-07-21T18:15:58.938855858Z ovnk_database_store/ovnkube-node-v767z_sbdb
2026-07-21T18:15:58.962075041Z ovnk_database_store/ovnkube-node-dq7d6_sbdb
2026-07-21T18:15:58.980696684Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:15:58.992562613Z INFO: Network log collection complete.
2026-07-21T18:15:59.060927467Z INFO: Getting status from alertmanager-main-0
2026-07-21T18:15:59.065369293Z Wrote inspect data to must-gather.
2026-07-21T18:15:59.179146630Z Gathering data for ns/openshift-console-operator...
2026-07-21T18:15:59.248111803Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:15:59.251598749Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-21T18:15:59.396853101Z Wrote inspect data to must-gather.
2026-07-21T18:15:59.453565899Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:15:59.620800850Z Gathering data for ns/openshift-console...
2026-07-21T18:15:59.692706846Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:15:59.696862970Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-21T18:15:59.721641300Z Wrote inspect data to must-gather.
2026-07-21T18:15:59.927043197Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:15:59.994545970Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-21T18:15:59.997200296Z INFO: Fallback to identify the container image from release info
2026-07-21T18:16:00.150852119Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:16:00.153558163Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-21T18:16:00.195700330Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-21T18:16:00.333185632Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:16:00.531275044Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:16:00.533475726Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-21T18:16:00.729456007Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:16:00.735843375Z Gathering data for ns/openshift-dns-operator...
2026-07-21T18:16:00.741580352Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T18:16:00.768856499Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-21T18:16:00.914436977Z daemonset.apps/perf-node-gather-daemonset created
2026-07-21T18:16:00.976081543Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:16:00.979805027Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-21T18:16:01.091453378Z Waiting for performance profile collector pods to become ready: 1
2026-07-21T18:16:01.094518653Z Gathering data for ns/openshift-dns...
2026-07-21T18:16:01.168232244Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:16:01.367051462Z Wrote inspect data to /must-gather/istio.
2026-07-21T18:16:01.770074751Z Gathering data for ns/openshift-image-registry...
2026-07-21T18:16:02.081731982Z Done executing Istio gather script
2026-07-21T18:16:02.228415262Z Waiting for performance profile collector pods to become ready: 2
2026-07-21T18:16:02.262910195Z Gathering data for ns/openshift-ingress-operator...
2026-07-21T18:16:02.597112483Z Gathering data for ns/openshift-ingress...
2026-07-21T18:16:02.945385067Z Gathering data for ns/openshift-ingress-canary...
2026-07-21T18:16:03.341651712Z Gathering data for ns/openshift-insights...
2026-07-21T18:16:03.367569106Z Waiting for performance profile collector pods to become ready: 3
2026-07-21T18:16:04.482326550Z Waiting for performance profile collector pods to become ready: 4
2026-07-21T18:16:05.604236751Z Waiting for performance profile collector pods to become ready: 5
2026-07-21T18:16:05.662089839Z Gathering data for ns/kserve...
2026-07-21T18:16:05.748959739Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T18:16:06.718683941Z Waiting for performance profile collector pods to become ready: 6
2026-07-21T18:16:07.258553074Z Gathering data for ns/openshift-config...
2026-07-21T18:16:07.573173299Z Gathering data for ns/openshift-config-managed...
2026-07-21T18:16:07.837164183Z Waiting for performance profile collector pods to become ready: 7
2026-07-21T18:16:08.038369593Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-21T18:16:08.316824767Z Gathering data for ns/openshift-kube-apiserver...
2026-07-21T18:16:08.588657808Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-21T18:16:08.871963537Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-21T18:16:08.974929451Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-21T18:16:09.214656839Z Gathering data for ns/openshift-kube-scheduler...
2026-07-21T18:16:09.251326828Z Collecting performance related data for node ip-10-0-135-98.ec2.internal
2026-07-21T18:16:09.262785336Z Collecting performance related data for node ip-10-0-136-21.ec2.internal
2026-07-21T18:16:09.275783804Z Collecting performance related data for node ip-10-0-132-249.ec2.internal
2026-07-21T18:16:09.543897172Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-21T18:16:09.927846381Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-21T18:16:10.332436115Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-21T18:16:10.745227300Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-21T18:16:10.755860067Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T18:16:11.327376568Z Gathering data for ns/openshift-multus...
2026-07-21T18:16:12.792876478Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-21T18:16:14.060179562Z Gathering data for ns/openshift-host-network...
2026-07-21T18:16:14.415486726Z Gathering data for ns/openshift-network-diagnostics...
2026-07-21T18:16:14.970910023Z Gathering data for ns/openshift-network-node-identity...
2026-07-21T18:16:15.418795375Z Gathering data for ns/openshift-network-operator...
2026-07-21T18:16:15.762920745Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T18:16:15.813894924Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-21T18:16:16.082172058Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-21T18:16:16.576351212Z Gathering data for ns/openshift-apiserver-operator...
2026-07-21T18:16:16.878907973Z Gathering data for ns/openshift-apiserver...
2026-07-21T18:16:17.287555797Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-21T18:16:17.555203358Z Collecting kubelet logs for node ip-10-0-135-98.ec2.internal
2026-07-21T18:16:17.558301146Z Collecting kubelet logs for node ip-10-0-136-21.ec2.internal
2026-07-21T18:16:17.561323396Z Collecting kubelet logs for node ip-10-0-132-249.ec2.internal
2026-07-21T18:16:17.617021172Z Gathering data for ns/openshift-controller-manager...
2026-07-21T18:16:17.966072687Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-21T18:16:17.975108528Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-p4dfr namespace
2026-07-21T18:16:17.979918387Z INFO: Node performance data collection complete.
2026-07-21T18:16:18.628552883Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-21T18:16:18.908496126Z Gathering data for ns/openshift-service-ca-operator...
2026-07-21T18:16:19.231719825Z Gathering data for ns/openshift-service-ca...
2026-07-21T18:16:19.602304709Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-21T18:16:20.472251040Z Wrote inspect data to must-gather.
2026-07-21T18:16:20.472301349Z error: inspection completed with the errors occurred while gathering data:
2026-07-21T18:16:20.472301349Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-21T18:16:20.601525816Z error: the server doesn't have a resource type "clusters"
2026-07-21T18:16:20.736351758Z Caches written to disk