2026-09-15T10:08:49.499217943Z [disk usage checker] Started
2026-09-15T10:08:49.502951735Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-15T10:08:49.684914912Z Gathering data for ns/openshift-cluster-version...
2026-09-15T10:08:49.812415967Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-15T10:08:49.894179036Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-15T10:08:49.971814112Z Gathering data for ns/default...
2026-09-15T10:08:50.250294119Z Gathering data for ns/openshift...
2026-09-15T10:08:50.642997456Z Gathering data for ns/kube-system...
2026-09-15T10:08:51.281250726Z Gathering data for ns/openshift-etcd...
2026-09-15T10:08:51.562020714Z Wrote inspect data to must-gather.
2026-09-15T10:08:51.562053175Z error: inspection completed with the errors occurred while gathering data:
2026-09-15T10:08:51.562053175Z     namespaces "assisted-installer" not found
2026-09-15T10:08:52.114104124Z Waiting on subprocesses to finish execution.
2026-09-15T10:08:52.124093978Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-15T10:08:52.124131335Z INFO: Gathering HAProxy config files
2026-09-15T10:08:52.127052162Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-15T10:08:52.131510501Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-15T10:08:52.161863355Z Executing Istio gather script
2026-09-15T10:08:52.202085769Z INFO: Waiting for node performance related collection to complete ...
2026-09-15T10:08:52.223576184Z INFO: Collecting host service logs for crio
2026-09-15T10:08:52.223943137Z INFO: Collecting host service logs for kubelet
2026-09-15T10:08:52.224303121Z INFO: Collecting host service logs for rpm-ostreed
2026-09-15T10:08:52.224655568Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-15T10:08:52.225006507Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-15T10:08:52.225362963Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-15T10:08:52.225710885Z INFO: Collecting host service logs for NetworkManager
2026-09-15T10:08:52.226043499Z INFO: Collecting host service logs for openvswitch
2026-09-15T10:08:52.226389530Z INFO: Collecting host service logs for ovs-configuration
2026-09-15T10:08:52.226734748Z INFO: Collecting host service logs for ovsdb-server
2026-09-15T10:08:52.227070132Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-15T10:08:52.227508152Z INFO: Waiting for worker host service log collection to complete ...
2026-09-15T10:08:52.596703283Z INFO: Collecting Insights Archives from insights-operator-5844b856db-kshh2
2026-09-15T10:08:52.596761393Z insights-runtime-extractor-4fkbb
2026-09-15T10:08:52.596782786Z insights-runtime-extractor-4ts6p
2026-09-15T10:08:52.596800610Z insights-runtime-extractor-nz7lt
2026-09-15T10:08:53.013156912Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-15T10:08:53.075829049Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-15T10:08:53.233744047Z error: the server doesn't have a resource type "performanceprofile"
2026-09-15T10:08:53.276522723Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-15T10:08:53.276522723Z See 'oc rsync -h' for help and examples
2026-09-15T10:08:53.408540576Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-15T10:08:53.408582693Z INFO: on-disk MachineConfig config collection complete.
2026-09-15T10:08:53.478748081Z No resources found
2026-09-15T10:08:53.499092041Z INFO: "metallb-operator" not detected. Skipping.
2026-09-15T10:08:53.525678545Z ERROR: No running kube-apiserver pods found
2026-09-15T10:08:53.545974202Z Wrote inspect data to must-gather.
2026-09-15T10:08:53.568482171Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-15T10:08:53.638910654Z No resources found in openshift-etcd namespace.
2026-09-15T10:08:53.844613298Z INFO: Worker host service log collection to complete.
2026-09-15T10:08:53.900100466Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-15T10:08:54.132252203Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-15T10:08:54.280068062Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-15T10:08:54.316905380Z Wrote inspect data to must-gather.
2026-09-15T10:08:54.359222426Z Wrote inspect data to must-gather.
2026-09-15T10:08:54.460578685Z Wrote inspect data to must-gather.
2026-09-15T10:08:54.469410489Z Gathering data for ns/openshift-monitoring...
2026-09-15T10:08:54.560701863Z No resources found
2026-09-15T10:08:54.581394514Z INFO: 'previous-logs' folder not found on ip-10-0-128-61.ec2.internal, skipping...
2026-09-15T10:08:54.619395167Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-15T10:08:54.665134695Z Wrote inspect data to must-gather.
2026-09-15T10:08:54.665738226Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-15T10:08:54.754396140Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-15T10:08:54.782032901Z INFO: HAProxy config collection complete.
2026-09-15T10:08:54.833141614Z error: the server doesn't have a resource type "machineconfigs"
2026-09-15T10:08:54.874072901Z Wrote inspect data to must-gather.
2026-09-15T10:08:54.972366883Z Wrote inspect data to must-gather.
2026-09-15T10:08:55.054553920Z Wrote inspect data to must-gather.
2026-09-15T10:08:55.061699926Z INFO: 'previous-logs' folder not found on ip-10-0-138-52.ec2.internal, skipping...
2026-09-15T10:08:55.073406629Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-15T10:08:55.308455821Z INFO: Getting rules from prometheus-k8s-0
2026-09-15T10:08:55.346569860Z Wrote inspect data to must-gather.
2026-09-15T10:08:55.355477702Z INFO: 'previous-logs' folder not found on ip-10-0-143-166.ec2.internal, skipping...
2026-09-15T10:08:55.355508564Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-15T10:08:55.355544069Z INFO: Machine Config Daemon termination log collection complete.
2026-09-15T10:08:55.363640872Z Wrote inspect data to must-gather.
2026-09-15T10:08:55.516820813Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-15T10:08:55.646028566Z Wrote inspect data to must-gather.
2026-09-15T10:08:55.670987865Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-15T10:08:55.731528043Z INFO: Getting status/config from prometheus-k8s-0
2026-09-15T10:08:55.745096538Z Wrote inspect data to must-gather.
2026-09-15T10:08:55.977136793Z INFO: INTERCONNECT MODE
2026-09-15T10:08:55.977169338Z INFO: Gathering ovn-kubernetes DBs
2026-09-15T10:08:56.000275320Z Wrote inspect data to must-gather.
2026-09-15T10:08:56.039610110Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-15T10:08:56.154166332Z INFO: Gathering OVN_Northbound from ovnkube-node-6qqnf...
2026-09-15T10:08:56.154420960Z INFO: Gathering OVN_Northbound from ovnkube-node-l9ttf...
2026-09-15T10:08:56.154724820Z INFO: Gathering OVN_Northbound from ovnkube-node-x75bm...
2026-09-15T10:08:56.155088646Z INFO: Gathering OVN_Southbound from ovnkube-node-6qqnf...
2026-09-15T10:08:56.155318063Z INFO: Gathering OVN_Southbound from ovnkube-node-l9ttf...
2026-09-15T10:08:56.155609897Z INFO: Gathering OVN_Southbound from ovnkube-node-x75bm...
2026-09-15T10:08:56.206477574Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-15T10:08:56.253311732Z Wrote inspect data to must-gather.
2026-09-15T10:08:56.397917153Z tar: Removing leading `/' from member names
2026-09-15T10:08:56.473144138Z tar: Removing leading `/' from member names
2026-09-15T10:08:56.492940821Z tar: Removing leading `/' from member names
2026-09-15T10:08:56.509518972Z tar: Removing leading `/' from member names
2026-09-15T10:08:56.546697421Z tar: Removing leading `/' from member names
2026-09-15T10:08:56.557269224Z tar: Removing leading `/' from member names
2026-09-15T10:08:56.645189094Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-15T10:08:56.653533003Z Wrote inspect data to must-gather.
2026-09-15T10:08:56.880616805Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-15T10:08:56.914265315Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-15T10:08:57.028913702Z Wrote inspect data to must-gather.
2026-09-15T10:08:57.076823457Z INFO: Waiting for network log collection to complete ...
2026-09-15T10:08:57.076941088Z INFO: Waiting for ovnk database copies to complete ...
2026-09-15T10:08:57.077483967Z INFO: Copying ovnk databases complete.
2026-09-15T10:08:57.080907365Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-15T10:08:57.083869931Z ovnk_database_store/
2026-09-15T10:08:57.083869931Z ovnk_database_store/ovnkube-node-x75bm_sbdb
2026-09-15T10:08:57.091922321Z Gathering data for ns/openshift-network-console...
2026-09-15T10:08:57.101002476Z ovnk_database_store/ovnkube-node-6qqnf_sbdb
2026-09-15T10:08:57.106085133Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-15T10:08:57.116292667Z ovnk_database_store/ovnkube-node-x75bm_nbdb
2026-09-15T10:08:57.122515321Z ovnk_database_store/ovnkube-node-l9ttf_nbdb
2026-09-15T10:08:57.132124185Z ovnk_database_store/ovnkube-node-6qqnf_nbdb
2026-09-15T10:08:57.140437300Z ovnk_database_store/ovnkube-node-l9ttf_sbdb
2026-09-15T10:08:57.164851824Z INFO: Network log collection complete.
2026-09-15T10:08:57.176367166Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-15T10:08:57.311817242Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:57.359270191Z Wrote inspect data to must-gather.
2026-09-15T10:08:57.556622998Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:57.561377424Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-15T10:08:57.591257479Z Gathering data for ns/openshift-console-operator...
2026-09-15T10:08:57.596234104Z INFO: Getting status from alertmanager-main-0
2026-09-15T10:08:57.719984193Z Wrote inspect data to must-gather.
2026-09-15T10:08:57.781973701Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:58.003876484Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-15T10:08:58.007198194Z INFO: Fallback to identify the container image from release info
2026-09-15T10:08:58.011849489Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:58.016610856Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-15T10:08:58.027679133Z Gathering data for ns/openshift-console...
2026-09-15T10:08:58.210057054Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:58.389923534Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:58.394814520Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-15T10:08:58.531140187Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-15T10:08:58.627196185Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:58.812191273Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:58.814499348Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-15T10:08:59.001064927Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:59.021623349Z Gathering data for ns/openshift-dns-operator...
2026-09-15T10:08:59.189980075Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:59.192413285Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-15T10:08:59.366753974Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:59.390946411Z Gathering data for ns/openshift-dns...
2026-09-15T10:08:59.544939177Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-15T10:08:59.545047516Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:59.547218852Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-15T10:08:59.674661551Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-15T10:08:59.685988673Z daemonset.apps/perf-node-gather-daemonset created
2026-09-15T10:08:59.722786071Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:08:59.828031363Z Waiting for performance profile collector pods to become ready: 1
2026-09-15T10:08:59.927176314Z Wrote inspect data to /must-gather/istio.
2026-09-15T10:09:00.019124073Z Gathering data for ns/openshift-image-registry...
2026-09-15T10:09:00.501621311Z Gathering data for ns/openshift-ingress-operator...
2026-09-15T10:09:00.632637344Z Done executing Istio gather script
2026-09-15T10:09:00.845492293Z Gathering data for ns/openshift-ingress...
2026-09-15T10:09:00.957993535Z Waiting for performance profile collector pods to become ready: 2
2026-09-15T10:09:01.155561796Z Gathering data for ns/openshift-ingress-canary...
2026-09-15T10:09:01.611734828Z Gathering data for ns/openshift-insights...
2026-09-15T10:09:02.086281242Z Waiting for performance profile collector pods to become ready: 3
2026-09-15T10:09:03.216602400Z Waiting for performance profile collector pods to become ready: 4
2026-09-15T10:09:03.986515450Z Gathering data for ns/kserve...
2026-09-15T10:09:04.347916144Z Waiting for performance profile collector pods to become ready: 5
2026-09-15T10:09:04.679626032Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-15T10:09:05.283944067Z Gathering data for ns/openshift-config...
2026-09-15T10:09:05.465562483Z Waiting for performance profile collector pods to become ready: 6
2026-09-15T10:09:05.541239770Z Gathering data for ns/openshift-config-managed...
2026-09-15T10:09:05.933873018Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-15T10:09:06.164836950Z Gathering data for ns/openshift-kube-apiserver...
2026-09-15T10:09:06.389356029Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-15T10:09:06.586713302Z Waiting for performance profile collector pods to become ready: 7
2026-09-15T10:09:06.655521795Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-15T10:09:06.946219748Z Gathering data for ns/openshift-kube-scheduler...
2026-09-15T10:09:07.167288742Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-15T10:09:07.501643950Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-15T10:09:07.703256464Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-15T10:09:07.806104093Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-15T10:09:07.964797579Z Collecting performance related data for node ip-10-0-128-61.ec2.internal
2026-09-15T10:09:07.971885834Z Collecting performance related data for node ip-10-0-138-52.ec2.internal
2026-09-15T10:09:07.981097738Z Collecting performance related data for node ip-10-0-143-166.ec2.internal
2026-09-15T10:09:08.155924583Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-15T10:09:08.720056738Z Gathering data for ns/openshift-multus...
2026-09-15T10:09:09.684131966Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-15T10:09:10.052519438Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-15T10:09:11.299082748Z Gathering data for ns/openshift-host-network...
2026-09-15T10:09:11.598317480Z Gathering data for ns/openshift-network-diagnostics...
2026-09-15T10:09:12.131258635Z Gathering data for ns/openshift-network-node-identity...
2026-09-15T10:09:12.497494362Z Gathering data for ns/openshift-network-operator...
2026-09-15T10:09:12.937984994Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-15T10:09:13.176533912Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-15T10:09:13.605836637Z Gathering data for ns/openshift-apiserver-operator...
2026-09-15T10:09:13.911783713Z Gathering data for ns/openshift-apiserver...
2026-09-15T10:09:14.253858573Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-15T10:09:14.502791195Z Gathering data for ns/openshift-controller-manager...
2026-09-15T10:09:14.688625500Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-09-15T10:09:14.773080360Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-15T10:09:15.390718709Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-15T10:09:15.709186780Z Gathering data for ns/openshift-service-ca-operator...
2026-09-15T10:09:15.986383286Z Gathering data for ns/openshift-service-ca...
2026-09-15T10:09:16.116621460Z Collecting kubelet logs for node ip-10-0-128-61.ec2.internal
2026-09-15T10:09:16.118249414Z Collecting kubelet logs for node ip-10-0-138-52.ec2.internal
2026-09-15T10:09:16.120029107Z Collecting kubelet logs for node ip-10-0-143-166.ec2.internal
2026-09-15T10:09:16.379620199Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-15T10:09:16.484553811Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5xh8r namespace
2026-09-15T10:09:16.488592675Z INFO: Node performance data collection complete.
2026-09-15T10:09:17.216234556Z Wrote inspect data to must-gather.
2026-09-15T10:09:17.216276567Z error: inspection completed with the errors occurred while gathering data:
2026-09-15T10:09:17.216276567Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-15T10:09:17.343594986Z error: the server doesn't have a resource type "clusters"
2026-09-15T10:09:17.481548895Z Caches written to disk