2026-09-13T17:55:56.180456732Z [disk usage checker] Started
2026-09-13T17:55:56.184015728Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-13T17:55:56.367516603Z Gathering data for ns/openshift-cluster-version...
2026-09-13T17:55:56.486360389Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-13T17:55:56.561183232Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-13T17:55:56.633460439Z Gathering data for ns/default...
2026-09-13T17:55:56.892698307Z Gathering data for ns/openshift...
2026-09-13T17:55:57.264919560Z Gathering data for ns/kube-system...
2026-09-13T17:55:57.879638384Z Gathering data for ns/openshift-etcd...
2026-09-13T17:55:58.146048304Z Wrote inspect data to must-gather.
2026-09-13T17:55:58.146088544Z error: inspection completed with the errors occurred while gathering data:
2026-09-13T17:55:58.146088544Z     namespaces "assisted-installer" not found
2026-09-13T17:55:58.656858753Z Waiting on subprocesses to finish execution.
2026-09-13T17:55:58.661432170Z INFO: Gathering HAProxy config files
2026-09-13T17:55:58.670104461Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-13T17:55:58.674019397Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-13T17:55:58.677098897Z Executing Istio gather script
2026-09-13T17:55:58.678977993Z INFO: Collecting host service logs for crio
2026-09-13T17:55:58.679050432Z INFO: Waiting for node performance related collection to complete ...
2026-09-13T17:55:58.679311575Z INFO: Collecting host service logs for kubelet
2026-09-13T17:55:58.679647278Z INFO: Collecting host service logs for rpm-ostreed
2026-09-13T17:55:58.680014093Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-13T17:55:58.680481114Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-13T17:55:58.680822175Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-13T17:55:58.681169079Z INFO: Collecting host service logs for NetworkManager
2026-09-13T17:55:58.681585338Z INFO: Collecting host service logs for openvswitch
2026-09-13T17:55:58.681974669Z INFO: Collecting host service logs for ovs-configuration
2026-09-13T17:55:58.682337069Z INFO: Collecting host service logs for ovsdb-server
2026-09-13T17:55:58.682645579Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-13T17:55:58.683057137Z INFO: Waiting for worker host service log collection to complete ...
2026-09-13T17:55:58.701384675Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-13T17:55:59.175665896Z No resources found in openshift-etcd namespace.
2026-09-13T17:55:59.366150606Z INFO: "metallb-operator" not detected. Skipping.
2026-09-13T17:55:59.444209157Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-13T17:55:59.464314206Z No resources found
2026-09-13T17:55:59.751105991Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-13T17:55:59.751192337Z INFO: on-disk MachineConfig config collection complete.
2026-09-13T17:55:59.793223859Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-13T17:55:59.816474449Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-13T17:56:00.027864704Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-13T17:56:00.055069555Z INFO: Worker host service log collection to complete.
2026-09-13T17:56:00.121831147Z Wrote inspect data to must-gather.
2026-09-13T17:56:00.242736150Z error: the server doesn't have a resource type "performanceprofile"
2026-09-13T17:56:00.251217362Z INFO: Collecting Insights Archives from insights-operator-5844b856db-gnl78
2026-09-13T17:56:00.251217362Z insights-runtime-extractor-c77dl
2026-09-13T17:56:00.251217362Z insights-runtime-extractor-d6xhd
2026-09-13T17:56:00.251217362Z insights-runtime-extractor-q9rnr
2026-09-13T17:56:00.349175264Z Wrote inspect data to must-gather.
2026-09-13T17:56:00.416849840Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-13T17:56:00.437291333Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-13T17:56:00.514042963Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-13T17:56:00.713633396Z tar: Removing leading `/' from member names
2026-09-13T17:56:00.718215014Z INFO: HAProxy config collection complete.
2026-09-13T17:56:00.725092989Z Wrote inspect data to must-gather.
2026-09-13T17:56:00.803713454Z No resources found
2026-09-13T17:56:00.809812393Z Gathering data for ns/openshift-monitoring...
2026-09-13T17:56:00.875391504Z ERROR: No running kube-apiserver pods found
2026-09-13T17:56:00.928618612Z Wrote inspect data to must-gather.
2026-09-13T17:56:00.936056764Z Wrote inspect data to must-gather.
2026-09-13T17:56:00.980198652Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-13T17:56:00.980198652Z See 'oc rsync -h' for help and examples
2026-09-13T17:56:01.021889235Z Wrote inspect data to must-gather.
2026-09-13T17:56:01.033560580Z Wrote inspect data to must-gather.
2026-09-13T17:56:01.081316798Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-13T17:56:01.158627955Z INFO: 'previous-logs' folder not found on ip-10-0-129-100.ec2.internal, skipping...
2026-09-13T17:56:01.194010278Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-13T17:56:01.194659530Z Wrote inspect data to must-gather.
2026-09-13T17:56:01.252387253Z error: the server doesn't have a resource type "machineconfigs"
2026-09-13T17:56:01.331320256Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-13T17:56:01.429401879Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-13T17:56:01.470732163Z INFO: 'previous-logs' folder not found on ip-10-0-141-239.ec2.internal, skipping...
2026-09-13T17:56:01.473464429Z Wrote inspect data to must-gather.
2026-09-13T17:56:01.526749929Z INFO: Getting rules from prometheus-k8s-0
2026-09-13T17:56:01.691104409Z Wrote inspect data to must-gather.
2026-09-13T17:56:01.729207364Z Wrote inspect data to must-gather.
2026-09-13T17:56:01.730900876Z INFO: 'previous-logs' folder not found on ip-10-0-143-240.ec2.internal, skipping...
2026-09-13T17:56:01.730932397Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-13T17:56:01.730974335Z INFO: Machine Config Daemon termination log collection complete.
2026-09-13T17:56:01.841506943Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-13T17:56:01.931411293Z INFO: Getting status/config from prometheus-k8s-0
2026-09-13T17:56:02.025208235Z INFO: INTERCONNECT MODE
2026-09-13T17:56:02.025244333Z INFO: Gathering ovn-kubernetes DBs
2026-09-13T17:56:02.070326520Z Wrote inspect data to must-gather.
2026-09-13T17:56:02.144502221Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-13T17:56:02.206226976Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-13T17:56:02.208494545Z INFO: Gathering OVN_Northbound from ovnkube-node-dbbsz...
2026-09-13T17:56:02.208739701Z INFO: Gathering OVN_Northbound from ovnkube-node-jqhtg...
2026-09-13T17:56:02.209011404Z INFO: Gathering OVN_Northbound from ovnkube-node-lx6bz...
2026-09-13T17:56:02.209288686Z INFO: Gathering OVN_Southbound from ovnkube-node-dbbsz...
2026-09-13T17:56:02.209601790Z INFO: Gathering OVN_Southbound from ovnkube-node-jqhtg...
2026-09-13T17:56:02.209897781Z INFO: Gathering OVN_Southbound from ovnkube-node-lx6bz...
2026-09-13T17:56:02.383726411Z Wrote inspect data to must-gather.
2026-09-13T17:56:02.395309412Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:02.509375325Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-13T17:56:02.545806317Z tar: Removing leading `/' from member names
2026-09-13T17:56:02.571118578Z tar: Removing leading `/' from member names
2026-09-13T17:56:02.589145957Z tar: Removing leading `/' from member names
2026-09-13T17:56:02.593337177Z tar: Removing leading `/' from member names
2026-09-13T17:56:02.597649295Z tar: Removing leading `/' from member names
2026-09-13T17:56:02.632226199Z tar: Removing leading `/' from member names
2026-09-13T17:56:02.696049716Z Wrote inspect data to must-gather.
2026-09-13T17:56:02.719463974Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:02.722430578Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-13T17:56:02.774801695Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-13T17:56:02.941585225Z W0913 17:56:02.941532    1392 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-13T17:56:02.949526067Z Wrote inspect data to must-gather.
2026-09-13T17:56:02.956112213Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:02.995320414Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-13T17:56:03.022299244Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-13T17:56:03.174723800Z INFO: Waiting for network log collection to complete ...
2026-09-13T17:56:03.174760622Z INFO: Waiting for ovnk database copies to complete ...
2026-09-13T17:56:03.175341996Z INFO: Copying ovnk databases complete.
2026-09-13T17:56:03.177071023Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-09-13T17:56:03.179993584Z ovnk_database_store/
2026-09-13T17:56:03.180046532Z ovnk_database_store/ovnkube-node-dbbsz_nbdb
2026-09-13T17:56:03.188587886Z ovnk_database_store/ovnkube-node-dbbsz_sbdb
2026-09-13T17:56:03.196149994Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:03.199096032Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-13T17:56:03.203538604Z ovnk_database_store/ovnkube-node-jqhtg_nbdb
2026-09-13T17:56:03.208190727Z Gathering data for ns/openshift-network-console...
2026-09-13T17:56:03.209852833Z ovnk_database_store/ovnkube-node-jqhtg_sbdb
2026-09-13T17:56:03.227092775Z ovnk_database_store/ovnkube-node-lx6bz_nbdb
2026-09-13T17:56:03.232167504Z ovnk_database_store/ovnkube-node-lx6bz_sbdb
2026-09-13T17:56:03.251428436Z INFO: Network log collection complete.
2026-09-13T17:56:03.278381902Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-13T17:56:03.283549124Z Wrote inspect data to must-gather.
2026-09-13T17:56:03.418932855Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:03.605076393Z Wrote inspect data to must-gather.
2026-09-13T17:56:03.639397534Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:03.642915995Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-13T17:56:03.654880287Z Gathering data for ns/openshift-console-operator...
2026-09-13T17:56:03.681226816Z INFO: Getting status from alertmanager-main-0
2026-09-13T17:56:03.886768628Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:03.921020869Z Wrote inspect data to must-gather.
2026-09-13T17:56:04.055256199Z Gathering data for ns/openshift-console...
2026-09-13T17:56:04.088852161Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:04.091914491Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-13T17:56:04.205750639Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-13T17:56:04.208563289Z INFO: Fallback to identify the container image from release info
2026-09-13T17:56:04.263035922Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:04.461598645Z Wrote inspect data to /must-gather/istio.
2026-09-13T17:56:04.523460361Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-13T17:56:04.714879698Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-13T17:56:04.841047515Z daemonset.apps/perf-node-gather-daemonset created
2026-09-13T17:56:04.987821469Z Waiting for performance profile collector pods to become ready: 1
2026-09-13T17:56:05.032221512Z Gathering data for ns/openshift-dns-operator...
2026-09-13T17:56:05.153379119Z Done executing Istio gather script
2026-09-13T17:56:05.289148166Z Gathering data for ns/openshift-dns...
2026-09-13T17:56:05.782579617Z Gathering data for ns/openshift-image-registry...
2026-09-13T17:56:06.109060613Z Waiting for performance profile collector pods to become ready: 2
2026-09-13T17:56:06.192556215Z Gathering data for ns/openshift-ingress-operator...
2026-09-13T17:56:06.200019599Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-13T17:56:06.465092133Z Gathering data for ns/openshift-ingress...
2026-09-13T17:56:06.736060054Z Gathering data for ns/openshift-ingress-canary...
2026-09-13T17:56:07.051216484Z Gathering data for ns/openshift-insights...
2026-09-13T17:56:07.215167408Z Waiting for performance profile collector pods to become ready: 3
2026-09-13T17:56:08.324095238Z Waiting for performance profile collector pods to become ready: 4
2026-09-13T17:56:08.752852384Z Gathering data for ns/openshift-config...
2026-09-13T17:56:08.963765422Z Gathering data for ns/openshift-config-managed...
2026-09-13T17:56:09.320250074Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-13T17:56:09.442835458Z Waiting for performance profile collector pods to become ready: 5
2026-09-13T17:56:09.542880534Z Gathering data for ns/openshift-kube-apiserver...
2026-09-13T17:56:09.752872232Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-13T17:56:09.987858984Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-13T17:56:10.218111564Z Gathering data for ns/openshift-kube-scheduler...
2026-09-13T17:56:10.430290138Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-13T17:56:10.562355964Z Waiting for performance profile collector pods to become ready: 6
2026-09-13T17:56:10.675283872Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-13T17:56:10.932860002Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-13T17:56:11.182881290Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-13T17:56:11.207070856Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-13T17:56:11.584014645Z Gathering data for ns/openshift-multus...
2026-09-13T17:56:11.661876153Z Waiting for performance profile collector pods to become ready: 7
2026-09-13T17:56:12.694154846Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-13T17:56:12.767745667Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-13T17:56:12.985076359Z Collecting performance related data for node ip-10-0-129-100.ec2.internal
2026-09-13T17:56:12.991967420Z Collecting performance related data for node ip-10-0-141-239.ec2.internal
2026-09-13T17:56:12.999462408Z Collecting performance related data for node ip-10-0-143-240.ec2.internal
2026-09-13T17:56:13.898508745Z Gathering data for ns/openshift-host-network...
2026-09-13T17:56:14.258846415Z Gathering data for ns/openshift-network-diagnostics...
2026-09-13T17:56:14.751094236Z Gathering data for ns/openshift-network-node-identity...
2026-09-13T17:56:15.096493032Z Gathering data for ns/openshift-network-operator...
2026-09-13T17:56:15.451398851Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-13T17:56:15.737030663Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-13T17:56:16.162019610Z Gathering data for ns/openshift-apiserver-operator...
2026-09-13T17:56:16.214726814Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-13T17:56:16.405389877Z Gathering data for ns/openshift-apiserver...
2026-09-13T17:56:16.730626860Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-13T17:56:16.955089822Z Gathering data for ns/openshift-controller-manager...
2026-09-13T17:56:17.194410392Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-13T17:56:17.791833919Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-13T17:56:18.041357593Z Gathering data for ns/openshift-service-ca-operator...
2026-09-13T17:56:18.303868565Z Gathering data for ns/openshift-service-ca...
2026-09-13T17:56:18.610641428Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-13T17:56:19.401048266Z Wrote inspect data to must-gather.
2026-09-13T17:56:19.401090813Z error: inspection completed with the errors occurred while gathering data:
2026-09-13T17:56:19.401090813Z     [skipping gathering secrets/support due to error: secrets "support" not found, skipping gathering daemonsets.apps/ovn-ipsec-containerized due to error: daemonsets.apps "ovn-ipsec-containerized" not found]
2026-09-13T17:56:20.863278260Z Collecting kubelet logs for node ip-10-0-129-100.ec2.internal
2026-09-13T17:56:20.865050253Z Collecting kubelet logs for node ip-10-0-141-239.ec2.internal
2026-09-13T17:56:20.866689896Z Collecting kubelet logs for node ip-10-0-143-240.ec2.internal
2026-09-13T17:56:21.166004225Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-gr5qm namespace
2026-09-13T17:56:21.170739295Z INFO: Node performance data collection complete.
2026-09-13T17:56:21.219425330Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-13T17:56:21.294618527Z error: the server doesn't have a resource type "clusters"
2026-09-13T17:56:21.452544167Z Caches written to disk