2026-09-11T07:50:44.394418578Z [disk usage checker] Started
2026-09-11T07:50:44.397719100Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-11T07:50:44.589871044Z Gathering data for ns/openshift-cluster-version...
2026-09-11T07:50:44.752633771Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T07:50:44.845073070Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T07:50:44.939627943Z Gathering data for ns/default...
2026-09-11T07:50:45.267060134Z Gathering data for ns/openshift...
2026-09-11T07:50:45.677656297Z Gathering data for ns/kube-system...
2026-09-11T07:50:46.360957875Z Gathering data for ns/openshift-etcd...
2026-09-11T07:50:46.654831758Z Wrote inspect data to must-gather.
2026-09-11T07:50:46.654863748Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T07:50:46.654863748Z     namespaces "assisted-installer" not found
2026-09-11T07:50:46.915359809Z Waiting on subprocesses to finish execution.
2026-09-11T07:50:46.923151066Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-11T07:50:46.928578024Z INFO: Gathering HAProxy config files
2026-09-11T07:50:46.943316904Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-11T07:50:46.944465951Z Executing Istio gather script
2026-09-11T07:50:46.946645476Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-11T07:50:46.960998717Z INFO: Collecting host service logs for crio
2026-09-11T07:50:46.961254309Z INFO: Collecting host service logs for kubelet
2026-09-11T07:50:46.961521364Z INFO: Collecting host service logs for rpm-ostreed
2026-09-11T07:50:46.961775165Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-11T07:50:46.962037605Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-11T07:50:46.962316109Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-11T07:50:46.962572364Z INFO: Collecting host service logs for NetworkManager
2026-09-11T07:50:46.962834193Z INFO: Collecting host service logs for openvswitch
2026-09-11T07:50:46.963122590Z INFO: Collecting host service logs for ovs-configuration
2026-09-11T07:50:46.963439501Z INFO: Collecting host service logs for ovsdb-server
2026-09-11T07:50:46.963728931Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-11T07:50:46.964074044Z INFO: Waiting for worker host service log collection to complete ...
2026-09-11T07:50:47.120195883Z INFO: Waiting for node performance related collection to complete ...
2026-09-11T07:50:47.403758808Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-11T07:50:47.455533857Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-11T07:50:47.460587669Z INFO: Collecting Insights Archives from insights-operator-5844b856db-ddggj
2026-09-11T07:50:47.460636847Z insights-runtime-extractor-fgspj
2026-09-11T07:50:47.460657612Z insights-runtime-extractor-lqg8q
2026-09-11T07:50:47.460676304Z insights-runtime-extractor-xlppx
2026-09-11T07:50:47.586841563Z No resources found
2026-09-11T07:50:47.798313276Z INFO: "metallb-operator" not detected. Skipping.
2026-09-11T07:50:48.011728112Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-11T07:50:48.165092057Z No resources found in openshift-etcd namespace.
2026-09-11T07:50:48.182343165Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-11T07:50:48.182343165Z See 'oc rsync -h' for help and examples
2026-09-11T07:50:48.184379341Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-11T07:50:48.540600755Z Wrote inspect data to must-gather.
2026-09-11T07:50:48.569075418Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-11T07:50:48.569172448Z INFO: on-disk MachineConfig config collection complete.
2026-09-11T07:50:48.640441845Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-11T07:50:48.689194743Z INFO: Worker host service log collection to complete.
2026-09-11T07:50:48.731495482Z ERROR: No running kube-apiserver pods found
2026-09-11T07:50:48.812829787Z error: the server doesn't have a resource type "performanceprofile"
2026-09-11T07:50:48.827009632Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-11T07:50:48.993831447Z No resources found
2026-09-11T07:50:49.109628276Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-11T07:50:49.163543002Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T07:50:49.225372339Z Wrote inspect data to must-gather.
2026-09-11T07:50:49.287816090Z Wrote inspect data to must-gather.
2026-09-11T07:50:49.302807925Z Wrote inspect data to must-gather.
2026-09-11T07:50:49.372759555Z Wrote inspect data to must-gather.
2026-09-11T07:50:49.392713709Z Wrote inspect data to must-gather.
2026-09-11T07:50:49.415006387Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-11T07:50:49.431782084Z INFO: 'previous-logs' folder not found on ip-10-0-130-29.ec2.internal, skipping...
2026-09-11T07:50:49.507517028Z Wrote inspect data to must-gather.
2026-09-11T07:50:49.535699394Z tar: Removing leading `/' from member names
2026-09-11T07:50:49.540311747Z INFO: HAProxy config collection complete.
2026-09-11T07:50:49.636175111Z INFO: Getting rules from prometheus-k8s-0
2026-09-11T07:50:49.726856536Z Wrote inspect data to must-gather.
2026-09-11T07:50:49.749590551Z error: the server doesn't have a resource type "machineconfigs"
2026-09-11T07:50:49.766914373Z INFO: 'previous-logs' folder not found on ip-10-0-141-27.ec2.internal, skipping...
2026-09-11T07:50:49.798038522Z Gathering data for ns/openshift-monitoring...
2026-09-11T07:50:49.935815248Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-11T07:50:49.972459449Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-11T07:50:50.035689373Z Wrote inspect data to must-gather.
2026-09-11T07:50:50.059948877Z INFO: 'previous-logs' folder not found on ip-10-0-142-209.ec2.internal, skipping...
2026-09-11T07:50:50.060074606Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-11T07:50:50.060129652Z INFO: Machine Config Daemon termination log collection complete.
2026-09-11T07:50:50.092614516Z INFO: Getting status/config from prometheus-k8s-0
2026-09-11T07:50:50.163756350Z Wrote inspect data to must-gather.
2026-09-11T07:50:50.285731302Z Wrote inspect data to must-gather.
2026-09-11T07:50:50.326000287Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-11T07:50:50.508674108Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-11T07:50:50.529962293Z Wrote inspect data to must-gather.
2026-09-11T07:50:50.581179929Z INFO: INTERCONNECT MODE
2026-09-11T07:50:50.581251563Z INFO: Gathering ovn-kubernetes DBs
2026-09-11T07:50:50.726525951Z Wrote inspect data to must-gather.
2026-09-11T07:50:50.738963234Z INFO: Gathering OVN_Northbound from ovnkube-node-4g2c5...
2026-09-11T07:50:50.739260551Z INFO: Gathering OVN_Northbound from ovnkube-node-fv6b2...
2026-09-11T07:50:50.739529622Z INFO: Gathering OVN_Northbound from ovnkube-node-n4bpf...
2026-09-11T07:50:50.739796129Z INFO: Gathering OVN_Southbound from ovnkube-node-4g2c5...
2026-09-11T07:50:50.740251063Z INFO: Gathering OVN_Southbound from ovnkube-node-fv6b2...
2026-09-11T07:50:50.740555511Z INFO: Gathering OVN_Southbound from ovnkube-node-n4bpf...
2026-09-11T07:50:50.743667964Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-11T07:50:51.021136022Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-11T07:50:51.058518374Z tar: Removing leading `/' from member names
2026-09-11T07:50:51.077330133Z tar: Removing leading `/' from member names
2026-09-11T07:50:51.080715052Z tar: Removing leading `/' from member names
2026-09-11T07:50:51.113865390Z tar: Removing leading `/' from member names
2026-09-11T07:50:51.128288309Z Wrote inspect data to must-gather.
2026-09-11T07:50:51.129448816Z tar: Removing leading `/' from member names
2026-09-11T07:50:51.145174004Z tar: Removing leading `/' from member names
2026-09-11T07:50:51.194561906Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-11T07:50:51.248330082Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:51.405384943Z Wrote inspect data to must-gather.
2026-09-11T07:50:51.408269644Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T07:50:51.495713935Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-11T07:50:51.496518315Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:51.499768205Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-11T07:50:51.570612986Z INFO: Waiting for network log collection to complete ...
2026-09-11T07:50:51.570663552Z INFO: Waiting for ovnk database copies to complete ...
2026-09-11T07:50:51.571143590Z INFO: Copying ovnk databases complete.
2026-09-11T07:50:51.572790662Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-11T07:50:51.574943983Z ovnk_database_store/
2026-09-11T07:50:51.574943983Z ovnk_database_store/ovnkube-node-4g2c5_sbdb
2026-09-11T07:50:51.596127471Z ovnk_database_store/ovnkube-node-n4bpf_nbdb
2026-09-11T07:50:51.603742604Z ovnk_database_store/ovnkube-node-n4bpf_sbdb
2026-09-11T07:50:51.620056131Z ovnk_database_store/ovnkube-node-fv6b2_nbdb
2026-09-11T07:50:51.628709966Z ovnk_database_store/ovnkube-node-4g2c5_nbdb
2026-09-11T07:50:51.635326234Z ovnk_database_store/ovnkube-node-fv6b2_sbdb
2026-09-11T07:50:51.637814271Z Wrote inspect data to must-gather.
2026-09-11T07:50:51.655469097Z INFO: Network log collection complete.
2026-09-11T07:50:51.728798585Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:51.850864588Z Wrote inspect data to must-gather.
2026-09-11T07:50:51.891296034Z INFO: Getting status from alertmanager-main-0
2026-09-11T07:50:51.950799947Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:51.953745890Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-11T07:50:52.053564712Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T07:50:52.126941241Z Wrote inspect data to must-gather.
2026-09-11T07:50:52.156376198Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:52.233295709Z Gathering data for ns/openshift-network-console...
2026-09-11T07:50:52.363148288Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:52.366107390Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-11T07:50:52.403270560Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-11T07:50:52.405103703Z INFO: Fallback to identify the container image from release info
2026-09-11T07:50:52.579857996Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:52.699529257Z Gathering data for ns/openshift-console-operator...
2026-09-11T07:50:52.767064921Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:52.769399682Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-11T07:50:52.924098762Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:53.116318793Z Gathering data for ns/openshift-console...
2026-09-11T07:50:53.119402418Z Wrote inspect data to /must-gather/istio.
2026-09-11T07:50:53.145213537Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-11T07:50:53.276082320Z daemonset.apps/perf-node-gather-daemonset created
2026-09-11T07:50:53.397385043Z Waiting for performance profile collector pods to become ready: 1
2026-09-11T07:50:53.671773099Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-11T07:50:53.798933391Z Done executing Istio gather script
2026-09-11T07:50:54.291726705Z Gathering data for ns/openshift-dns-operator...
2026-09-11T07:50:54.421837670Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-11T07:50:54.512412322Z Waiting for performance profile collector pods to become ready: 2
2026-09-11T07:50:54.580662191Z Gathering data for ns/openshift-dns...
2026-09-11T07:50:55.185144909Z Gathering data for ns/openshift-image-registry...
2026-09-11T07:50:55.634139434Z Waiting for performance profile collector pods to become ready: 3
2026-09-11T07:50:55.652069108Z Gathering data for ns/openshift-ingress-operator...
2026-09-11T07:50:55.959712684Z Gathering data for ns/openshift-ingress...
2026-09-11T07:50:56.265655003Z Gathering data for ns/openshift-ingress-canary...
2026-09-11T07:50:56.637495885Z Gathering data for ns/openshift-insights...
2026-09-11T07:50:56.734615362Z Waiting for performance profile collector pods to become ready: 4
2026-09-11T07:50:57.836447070Z Waiting for performance profile collector pods to become ready: 5
2026-09-11T07:50:58.466671416Z Gathering data for ns/openshift-jobset-operator...
2026-09-11T07:50:58.943595439Z Waiting for performance profile collector pods to become ready: 6
2026-09-11T07:50:59.248404767Z Gathering data for ns/openshift-config...
2026-09-11T07:50:59.427697009Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-11T07:50:59.498635283Z Gathering data for ns/openshift-config-managed...
2026-09-11T07:50:59.898275368Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-11T07:51:00.053894478Z Waiting for performance profile collector pods to become ready: 7
2026-09-11T07:51:00.162232087Z Gathering data for ns/openshift-kube-apiserver...
2026-09-11T07:51:00.455405856Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-11T07:51:00.724367214Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-11T07:51:01.005515203Z Gathering data for ns/openshift-kube-scheduler...
2026-09-11T07:51:01.156292619Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-11T07:51:01.279645481Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-11T07:51:01.409477109Z Collecting performance related data for node ip-10-0-142-209.ec2.internal
2026-09-11T07:51:01.417790536Z Collecting performance related data for node ip-10-0-141-27.ec2.internal
2026-09-11T07:51:01.425650872Z Collecting performance related data for node ip-10-0-130-29.ec2.internal
2026-09-11T07:51:01.619185456Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-11T07:51:01.994675663Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-11T07:51:02.388428526Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-11T07:51:03.015943625Z Gathering data for ns/openshift-multus...
2026-09-11T07:51:04.432490357Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-11T07:51:04.985173982Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-11T07:51:06.293163577Z Gathering data for ns/openshift-host-network...
2026-09-11T07:51:06.601174667Z Gathering data for ns/openshift-network-diagnostics...
2026-09-11T07:51:07.174386518Z Gathering data for ns/openshift-network-node-identity...
2026-09-11T07:51:07.710534934Z Gathering data for ns/openshift-network-operator...
2026-09-11T07:51:08.079661578Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-11T07:51:08.337773580Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-11T07:51:08.826466076Z Gathering data for ns/openshift-apiserver-operator...
2026-09-11T07:51:09.120704518Z Gathering data for ns/openshift-apiserver...
2026-09-11T07:51:09.438971482Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-11T07:51:09.504391307Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-11T07:51:09.545979347Z Collecting kubelet logs for node ip-10-0-142-209.ec2.internal
2026-09-11T07:51:09.548256528Z Collecting kubelet logs for node ip-10-0-141-27.ec2.internal
2026-09-11T07:51:09.550038696Z Collecting kubelet logs for node ip-10-0-130-29.ec2.internal
2026-09-11T07:51:09.815139221Z Gathering data for ns/openshift-controller-manager...
2026-09-11T07:51:09.895820996Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-crkcv namespace
2026-09-11T07:51:09.900952359Z INFO: Node performance data collection complete.
2026-09-11T07:51:10.141383269Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-11T07:51:10.763626426Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-11T07:51:11.163354163Z Gathering data for ns/openshift-service-ca-operator...
2026-09-11T07:51:11.458671367Z Gathering data for ns/openshift-service-ca...
2026-09-11T07:51:11.807450061Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-11T07:51:12.736406584Z Wrote inspect data to must-gather.
2026-09-11T07:51:12.736448421Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T07:51:12.736448421Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-11T07:51:12.857260982Z error: the server doesn't have a resource type "clusters"
2026-09-11T07:51:12.970743317Z Caches written to disk