2026-07-28T18:01:04.155947341Z [disk usage checker] Started
2026-07-28T18:01:04.159440234Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-07-28T18:01:04.360938876Z Gathering data for ns/openshift-cluster-version...
2026-07-28T18:01:04.504768149Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T18:01:04.604416512Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T18:01:04.693601624Z Gathering data for ns/default...
2026-07-28T18:01:05.004083415Z Gathering data for ns/openshift...
2026-07-28T18:01:05.420969555Z Gathering data for ns/kube-system...
2026-07-28T18:01:06.184583257Z Gathering data for ns/openshift-etcd...
2026-07-28T18:01:06.489568279Z Wrote inspect data to must-gather.
2026-07-28T18:01:06.489601528Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T18:01:06.489601528Z     namespaces "assisted-installer" not found
2026-07-28T18:01:06.952021563Z Waiting on subprocesses to finish execution.
2026-07-28T18:01:06.964852608Z Executing Istio gather script
2026-07-28T18:01:06.965133116Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-28T18:01:06.967916757Z INFO: Collecting host service logs for crio
2026-07-28T18:01:06.968281374Z INFO: Collecting host service logs for kubelet
2026-07-28T18:01:06.968628614Z INFO: Collecting host service logs for rpm-ostreed
2026-07-28T18:01:06.968918002Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-28T18:01:06.969246069Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-28T18:01:06.969352287Z INFO: Gathering HAProxy config files
2026-07-28T18:01:06.969546126Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-28T18:01:06.969848518Z INFO: Collecting host service logs for NetworkManager
2026-07-28T18:01:06.970152374Z INFO: Collecting host service logs for openvswitch
2026-07-28T18:01:06.970436185Z INFO: Collecting host service logs for ovs-configuration
2026-07-28T18:01:06.970730181Z INFO: Collecting host service logs for ovsdb-server
2026-07-28T18:01:06.971068618Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-28T18:01:06.971388718Z INFO: Waiting for worker host service log collection to complete ...
2026-07-28T18:01:06.973444858Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-28T18:01:06.980864501Z INFO: Waiting for node performance related collection to complete ...
2026-07-28T18:01:06.986858018Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-28T18:01:07.276733697Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-q5w5v
2026-07-28T18:01:07.276733697Z insights-runtime-extractor-22wdg
2026-07-28T18:01:07.276733697Z insights-runtime-extractor-s7zwq
2026-07-28T18:01:07.276733697Z insights-runtime-extractor-xg5vg
2026-07-28T18:01:07.605872869Z No resources found in openshift-etcd namespace.
2026-07-28T18:01:08.067169156Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-28T18:01:08.167286779Z INFO: "metallb-operator" not detected. Skipping.
2026-07-28T18:01:08.187616204Z No resources found
2026-07-28T18:01:08.210348210Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-28T18:01:08.210348210Z See 'oc rsync -h' for help and examples
2026-07-28T18:01:08.358198398Z ERROR: No running kube-apiserver pods found
2026-07-28T18:01:08.507959642Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-28T18:01:08.510810210Z error: the server doesn't have a resource type "performanceprofile"
2026-07-28T18:01:08.563170758Z INFO: Worker host service log collection to complete.
2026-07-28T18:01:08.675209315Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-28T18:01:08.675296728Z INFO: on-disk MachineConfig config collection complete.
2026-07-28T18:01:08.773980866Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-28T18:01:08.802185898Z Wrote inspect data to must-gather.
2026-07-28T18:01:08.821488617Z No resources found
2026-07-28T18:01:08.885548052Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-28T18:01:09.075792220Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-28T18:01:09.089458790Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T18:01:09.095418614Z Wrote inspect data to must-gather.
2026-07-28T18:01:09.168823907Z Wrote inspect data to must-gather.
2026-07-28T18:01:09.213239577Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-07-28T18:01:09.300158955Z Wrote inspect data to must-gather.
2026-07-28T18:01:09.343711568Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-28T18:01:09.423144613Z Wrote inspect data to must-gather.
2026-07-28T18:01:09.443268376Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-28T18:01:09.480312965Z INFO: 'previous-logs' folder not found on ip-10-0-134-14.ec2.internal, skipping...
2026-07-28T18:01:09.510804229Z Wrote inspect data to must-gather.
2026-07-28T18:01:09.540639193Z Gathering data for ns/openshift-monitoring...
2026-07-28T18:01:09.559636820Z Wrote inspect data to must-gather.
2026-07-28T18:01:09.656512101Z tar: Removing leading `/' from member names
2026-07-28T18:01:09.665291122Z INFO: HAProxy config collection complete.
2026-07-28T18:01:09.743567739Z error: the server doesn't have a resource type "machineconfigs"
2026-07-28T18:01:09.847777142Z INFO: 'previous-logs' folder not found on ip-10-0-136-80.ec2.internal, skipping...
2026-07-28T18:01:09.896441734Z Wrote inspect data to must-gather.
2026-07-28T18:01:09.936611542Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-28T18:01:09.959204356Z INFO: Getting rules from prometheus-k8s-0
2026-07-28T18:01:10.173452710Z INFO: 'previous-logs' folder not found on ip-10-0-139-193.ec2.internal, skipping...
2026-07-28T18:01:10.173490585Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-28T18:01:10.173519875Z INFO: Machine Config Daemon termination log collection complete.
2026-07-28T18:01:10.204112598Z Wrote inspect data to must-gather.
2026-07-28T18:01:10.211452285Z Wrote inspect data to must-gather.
2026-07-28T18:01:10.379507678Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-28T18:01:10.423244126Z INFO: Getting status/config from prometheus-k8s-0
2026-07-28T18:01:10.426151452Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-28T18:01:10.605860432Z Wrote inspect data to must-gather.
2026-07-28T18:01:10.673590862Z Wrote inspect data to must-gather.
2026-07-28T18:01:10.822926827Z Wrote inspect data to must-gather.
2026-07-28T18:01:10.859488877Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-28T18:01:10.986090225Z INFO: INTERCONNECT MODE
2026-07-28T18:01:10.986184097Z INFO: Gathering ovn-kubernetes DBs
2026-07-28T18:01:11.044364384Z Wrote inspect data to must-gather.
2026-07-28T18:01:11.126211361Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-28T18:01:11.135726403Z INFO: Gathering OVN_Northbound from ovnkube-node-4p62f...
2026-07-28T18:01:11.136098298Z INFO: Gathering OVN_Northbound from ovnkube-node-h6lk7...
2026-07-28T18:01:11.136359209Z INFO: Gathering OVN_Northbound from ovnkube-node-kxpf9...
2026-07-28T18:01:11.136641173Z INFO: Gathering OVN_Southbound from ovnkube-node-4p62f...
2026-07-28T18:01:11.136927036Z INFO: Gathering OVN_Southbound from ovnkube-node-h6lk7...
2026-07-28T18:01:11.137245561Z INFO: Gathering OVN_Southbound from ovnkube-node-kxpf9...
2026-07-28T18:01:11.349137463Z Wrote inspect data to must-gather.
2026-07-28T18:01:11.583577370Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T18:01:11.603054479Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-28T18:01:11.620487689Z tar: Removing leading `/' from member names
2026-07-28T18:01:11.628191942Z tar: Removing leading `/' from member names
2026-07-28T18:01:11.628941413Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-28T18:01:11.629253975Z tar: Removing leading `/' from member names
2026-07-28T18:01:11.645359442Z tar: Removing leading `/' from member names
2026-07-28T18:01:11.658710493Z tar: Removing leading `/' from member names
2026-07-28T18:01:11.667767340Z Wrote inspect data to must-gather.
2026-07-28T18:01:11.684022004Z tar: Removing leading `/' from member names
2026-07-28T18:01:11.851709115Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:11.931438889Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-28T18:01:11.937593456Z Wrote inspect data to must-gather.
2026-07-28T18:01:12.035588134Z INFO: Waiting for network log collection to complete ...
2026-07-28T18:01:12.035634915Z INFO: Waiting for ovnk database copies to complete ...
2026-07-28T18:01:12.036140869Z INFO: Copying ovnk databases complete.
2026-07-28T18:01:12.037708687Z 3.8M	must-gather/network_logs/ovnk_database_store
2026-07-28T18:01:12.039872901Z ovnk_database_store/
2026-07-28T18:01:12.039872901Z ovnk_database_store/ovnkube-node-4p62f_nbdb
2026-07-28T18:01:12.051304696Z ovnk_database_store/ovnkube-node-h6lk7_sbdb
2026-07-28T18:01:12.077477041Z ovnk_database_store/ovnkube-node-kxpf9_nbdb
2026-07-28T18:01:12.086395371Z ovnk_database_store/ovnkube-node-kxpf9_sbdb
2026-07-28T18:01:12.098320099Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:12.103368100Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-28T18:01:12.112821152Z ovnk_database_store/ovnkube-node-4p62f_sbdb
2026-07-28T18:01:12.134103048Z ovnk_database_store/ovnkube-node-h6lk7_nbdb
2026-07-28T18:01:12.148212228Z INFO: Network log collection complete.
2026-07-28T18:01:12.206710956Z W0728 18:01:12.206666    1512 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-28T18:01:12.211980617Z Wrote inspect data to must-gather.
2026-07-28T18:01:12.307885598Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T18:01:12.329216643Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:12.388703580Z INFO: Getting status from alertmanager-main-0
2026-07-28T18:01:12.522818713Z Gathering data for ns/openshift-network-console...
2026-07-28T18:01:12.538834392Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-28T18:01:12.540811108Z INFO: Fallback to identify the container image from release info
2026-07-28T18:01:12.560480216Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:12.562823060Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-28T18:01:12.772448059Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:12.980824357Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:12.983518983Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-28T18:01:13.030190095Z Gathering data for ns/openshift-console-operator...
2026-07-28T18:01:13.198586455Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:13.274266918Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-28T18:01:13.413462384Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:13.415647111Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-28T18:01:13.422719299Z daemonset.apps/perf-node-gather-daemonset created
2026-07-28T18:01:13.470441311Z Gathering data for ns/openshift-console...
2026-07-28T18:01:13.558290441Z Waiting for performance profile collector pods to become ready: 1
2026-07-28T18:01:13.606272039Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:13.794369126Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:13.797153966Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-28T18:01:14.003206616Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:14.083216547Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-28T18:01:14.214733520Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:14.217319104Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-28T18:01:14.225576989Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-07-28T18:01:14.382565500Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:14.606722490Z Wrote inspect data to /must-gather/istio.
2026-07-28T18:01:14.651258821Z Gathering data for ns/openshift-dns-operator...
2026-07-28T18:01:14.723793081Z Waiting for performance profile collector pods to become ready: 2
2026-07-28T18:01:14.997087489Z Gathering data for ns/openshift-dns...
2026-07-28T18:01:15.290929758Z Done executing Istio gather script
2026-07-28T18:01:15.687793709Z Gathering data for ns/openshift-image-registry...
2026-07-28T18:01:15.844304713Z Waiting for performance profile collector pods to become ready: 3
2026-07-28T18:01:16.234012471Z Gathering data for ns/openshift-ingress-operator...
2026-07-28T18:01:16.571306582Z Gathering data for ns/openshift-ingress...
2026-07-28T18:01:16.932378059Z Gathering data for ns/openshift-ingress-canary...
2026-07-28T18:01:16.979901100Z Waiting for performance profile collector pods to become ready: 4
2026-07-28T18:01:17.374443793Z Gathering data for ns/openshift-insights...
2026-07-28T18:01:18.107435960Z Waiting for performance profile collector pods to become ready: 5
2026-07-28T18:01:19.230770083Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-07-28T18:01:19.232077432Z Waiting for performance profile collector pods to become ready: 6
2026-07-28T18:01:19.812894015Z Gathering data for ns/kserve...
2026-07-28T18:01:20.343620271Z Waiting for performance profile collector pods to become ready: 7
2026-07-28T18:01:21.479536643Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-28T18:01:21.482148940Z Gathering data for ns/openshift-config...
2026-07-28T18:01:21.733795096Z Collecting performance related data for node ip-10-0-139-193.ec2.internal
2026-07-28T18:01:21.741407145Z Collecting performance related data for node ip-10-0-134-14.ec2.internal
2026-07-28T18:01:21.750431694Z Collecting performance related data for node ip-10-0-136-80.ec2.internal
2026-07-28T18:01:21.849297634Z Gathering data for ns/openshift-config-managed...
2026-07-28T18:01:22.390162766Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-28T18:01:22.722822750Z Gathering data for ns/openshift-kube-apiserver...
2026-07-28T18:01:23.093402171Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-28T18:01:23.475778726Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-28T18:01:23.825427467Z Gathering data for ns/openshift-kube-scheduler...
2026-07-28T18:01:24.199857064Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-28T18:01:24.237302158Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-07-28T18:01:24.554057634Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-28T18:01:24.954370665Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-28T18:01:25.304491061Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-28T18:01:25.963273220Z Gathering data for ns/openshift-multus...
2026-07-28T18:01:27.878968165Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-28T18:01:29.242066130Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-07-28T18:01:29.750975000Z Gathering data for ns/openshift-host-network...
2026-07-28T18:01:29.959214552Z Collecting kubelet logs for node ip-10-0-139-193.ec2.internal
2026-07-28T18:01:29.961490721Z Collecting kubelet logs for node ip-10-0-134-14.ec2.internal
2026-07-28T18:01:29.963506096Z Collecting kubelet logs for node ip-10-0-136-80.ec2.internal
2026-07-28T18:01:30.117927950Z Gathering data for ns/openshift-network-diagnostics...
2026-07-28T18:01:30.350609936Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ck55s namespace
2026-07-28T18:01:30.356386696Z INFO: Node performance data collection complete.
2026-07-28T18:01:30.785723104Z Gathering data for ns/openshift-network-node-identity...
2026-07-28T18:01:31.231368861Z Gathering data for ns/openshift-network-operator...
2026-07-28T18:01:31.684334071Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-28T18:01:32.017676704Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-28T18:01:32.609121193Z Gathering data for ns/openshift-apiserver-operator...
2026-07-28T18:01:32.927774018Z Gathering data for ns/openshift-apiserver...
2026-07-28T18:01:33.391072968Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-28T18:01:33.736066942Z Gathering data for ns/openshift-controller-manager...
2026-07-28T18:01:34.040284400Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-28T18:01:34.246420436Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-07-28T18:01:34.721266403Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-28T18:01:35.021195029Z Gathering data for ns/openshift-service-ca-operator...
2026-07-28T18:01:35.350482732Z Gathering data for ns/openshift-service-ca...
2026-07-28T18:01:35.759101854Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-28T18:01:36.756746954Z Wrote inspect data to must-gather.
2026-07-28T18:01:36.756792167Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T18:01:36.756792167Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-28T18:01:36.882220346Z error: the server doesn't have a resource type "clusters"
2026-07-28T18:01:37.061175486Z Caches written to disk