2026-05-26T06:52:42.974295248Z [disk usage checker] Started
2026-05-26T06:52:42.977941159Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-26T06:52:43.190449130Z Gathering data for ns/openshift-cluster-version...
2026-05-26T06:52:43.334446236Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T06:52:43.414843731Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T06:52:43.497404095Z Gathering data for ns/default...
2026-05-26T06:52:43.775390462Z Gathering data for ns/openshift...
2026-05-26T06:52:44.171505585Z Gathering data for ns/kube-system...
2026-05-26T06:52:44.854985042Z Gathering data for ns/openshift-etcd...
2026-05-26T06:52:45.135045026Z Wrote inspect data to must-gather.
2026-05-26T06:52:45.135076106Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T06:52:45.135076106Z     namespaces "assisted-installer" not found
2026-05-26T06:52:45.240688080Z Waiting on subprocesses to finish execution.
2026-05-26T06:52:45.247896185Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-26T06:52:45.254072949Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-26T06:52:45.254668870Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-26T06:52:45.256324560Z INFO: Gathering HAProxy config files
2026-05-26T06:52:45.270464320Z INFO: Collecting host service logs for crio
2026-05-26T06:52:45.270822767Z INFO: Collecting host service logs for kubelet
2026-05-26T06:52:45.271170120Z INFO: Collecting host service logs for rpm-ostreed
2026-05-26T06:52:45.271512749Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-26T06:52:45.271867078Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-26T06:52:45.272204252Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-26T06:52:45.272528691Z INFO: Collecting host service logs for NetworkManager
2026-05-26T06:52:45.272875127Z INFO: Collecting host service logs for openvswitch
2026-05-26T06:52:45.273209708Z INFO: Collecting host service logs for ovs-configuration
2026-05-26T06:52:45.273528772Z INFO: Collecting host service logs for ovsdb-server
2026-05-26T06:52:45.273909121Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-26T06:52:45.274280941Z INFO: Waiting for worker host service log collection to complete ...
2026-05-26T06:52:45.343423467Z INFO: Waiting for node performance related collection to complete ...
2026-05-26T06:52:45.504104567Z No resources found in openshift-etcd namespace.
2026-05-26T06:52:45.615635423Z No resources found
2026-05-26T06:52:45.935649733Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-26T06:52:45.935649733Z INFO: on-disk MachineConfig config collection complete.
2026-05-26T06:52:46.034069224Z error: the server doesn't have a resource type "performanceprofile"
2026-05-26T06:52:46.089139075Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-sftr9
2026-05-26T06:52:46.089192451Z insights-runtime-extractor-2kzrx
2026-05-26T06:52:46.089212922Z insights-runtime-extractor-mhjjw
2026-05-26T06:52:46.089232662Z insights-runtime-extractor-ttm5s
2026-05-26T06:52:46.097961885Z INFO: "metallb-operator" not detected. Skipping.
2026-05-26T06:52:46.170331118Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-26T06:52:46.280641649Z Wrote inspect data to must-gather.
2026-05-26T06:52:46.307317125Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-26T06:52:46.367142668Z ERROR: No running kube-apiserver pods found
2026-05-26T06:52:46.741468822Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-26T06:52:46.858968161Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T06:52:46.881847104Z Wrote inspect data to must-gather.
2026-05-26T06:52:46.950780001Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-26T06:52:46.992342780Z No resources found
2026-05-26T06:52:47.103938810Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-26T06:52:47.103938810Z See 'oc rsync -h' for help and examples
2026-05-26T06:52:47.131068508Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.218784549Z INFO: Worker host service log collection to complete.
2026-05-26T06:52:47.229233105Z INFO: 'previous-logs' folder not found on ip-10-0-132-73.ec2.internal, skipping...
2026-05-26T06:52:47.232266812Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-26T06:52:47.254098281Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-26T06:52:47.266345290Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.295113248Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-26T06:52:47.381295696Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.441865696Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.507939502Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.569400963Z error: the server doesn't have a resource type "machineconfigs"
2026-05-26T06:52:47.618797860Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.624836878Z INFO: 'previous-logs' folder not found on ip-10-0-135-20.ec2.internal, skipping...
2026-05-26T06:52:47.663830062Z INFO: Getting rules from prometheus-k8s-0
2026-05-26T06:52:47.673334888Z tar: Removing leading `/' from member names
2026-05-26T06:52:47.677355153Z INFO: HAProxy config collection complete.
2026-05-26T06:52:47.710836777Z Gathering data for ns/openshift-monitoring...
2026-05-26T06:52:47.801171677Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.865186325Z Wrote inspect data to must-gather.
2026-05-26T06:52:47.882797301Z INFO: 'previous-logs' folder not found on ip-10-0-136-226.ec2.internal, skipping...
2026-05-26T06:52:47.882826941Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-26T06:52:47.882837193Z INFO: Machine Config Daemon termination log collection complete.
2026-05-26T06:52:47.987421732Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-26T06:52:47.997395207Z Wrote inspect data to must-gather.
2026-05-26T06:52:48.093083715Z INFO: Getting status/config from prometheus-k8s-0
2026-05-26T06:52:48.099382984Z Wrote inspect data to must-gather.
2026-05-26T06:52:48.140285114Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-26T06:52:48.334174194Z Wrote inspect data to must-gather.
2026-05-26T06:52:48.353036434Z INFO: INTERCONNECT MODE
2026-05-26T06:52:48.353124315Z INFO: Gathering ovn-kubernetes DBs
2026-05-26T06:52:48.461972115Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-26T06:52:48.506219047Z INFO: Gathering OVN_Northbound from ovnkube-node-2hpf2...
2026-05-26T06:52:48.506482474Z INFO: Gathering OVN_Northbound from ovnkube-node-775ks...
2026-05-26T06:52:48.506693105Z INFO: Gathering OVN_Northbound from ovnkube-node-h8bsx...
2026-05-26T06:52:48.506962606Z INFO: Gathering OVN_Southbound from ovnkube-node-2hpf2...
2026-05-26T06:52:48.507224491Z INFO: Gathering OVN_Southbound from ovnkube-node-775ks...
2026-05-26T06:52:48.507464982Z INFO: Gathering OVN_Southbound from ovnkube-node-h8bsx...
2026-05-26T06:52:48.558055080Z Wrote inspect data to must-gather.
2026-05-26T06:52:48.743005184Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-26T06:52:48.807790701Z tar: Removing leading `/' from member names
2026-05-26T06:52:48.818031123Z tar: Removing leading `/' from member names
2026-05-26T06:52:48.828794654Z tar: Removing leading `/' from member names
2026-05-26T06:52:48.830003251Z tar: Removing leading `/' from member names
2026-05-26T06:52:48.839340552Z tar: Removing leading `/' from member names
2026-05-26T06:52:48.841811451Z tar: Removing leading `/' from member names
2026-05-26T06:52:48.864829400Z Wrote inspect data to must-gather.
2026-05-26T06:52:48.979346060Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-26T06:52:49.068335823Z Wrote inspect data to must-gather.
2026-05-26T06:52:49.213070961Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-26T06:52:49.246187802Z INFO: Waiting for network log collection to complete ...
2026-05-26T06:52:49.246222172Z INFO: Waiting for ovnk database copies to complete ...
2026-05-26T06:52:49.246689379Z INFO: Copying ovnk databases complete.
2026-05-26T06:52:49.248270758Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-05-26T06:52:49.251108740Z ovnk_database_store/
2026-05-26T06:52:49.251160921Z ovnk_database_store/ovnkube-node-775ks_sbdb
2026-05-26T06:52:49.273119235Z ovnk_database_store/ovnkube-node-775ks_nbdb
2026-05-26T06:52:49.282925175Z ovnk_database_store/ovnkube-node-h8bsx_sbdb
2026-05-26T06:52:49.286966373Z Wrote inspect data to must-gather.
2026-05-26T06:52:49.308697615Z ovnk_database_store/ovnkube-node-h8bsx_nbdb
2026-05-26T06:52:49.316684971Z ovnk_database_store/ovnkube-node-2hpf2_nbdb
2026-05-26T06:52:49.325099799Z ovnk_database_store/ovnkube-node-2hpf2_sbdb
2026-05-26T06:52:49.349541340Z INFO: Network log collection complete.
2026-05-26T06:52:49.374169673Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T06:52:49.465233042Z Wrote inspect data to must-gather.
2026-05-26T06:52:49.613892083Z INFO: Getting status from alertmanager-main-0
2026-05-26T06:52:49.644578601Z Wrote inspect data to must-gather.
2026-05-26T06:52:49.870817901Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-26T06:52:49.873066388Z INFO: Fallback to identify the container image from release info
2026-05-26T06:52:49.974222792Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T06:52:50.118054437Z Gathering data for ns/openshift-network-console...
2026-05-26T06:52:50.368015749Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-26T06:52:50.487686084Z daemonset.apps/perf-node-gather-daemonset created
2026-05-26T06:52:50.578312201Z Gathering data for ns/openshift-console-operator...
2026-05-26T06:52:50.609107655Z Waiting for performance profile collector pods to become ready: 1
2026-05-26T06:52:50.937315386Z Gathering data for ns/openshift-console...
2026-05-26T06:52:51.393319832Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-26T06:52:51.721569895Z Waiting for performance profile collector pods to become ready: 2
2026-05-26T06:52:51.759233156Z Gathering data for ns/openshift-dns-operator...
2026-05-26T06:52:52.018591280Z Gathering data for ns/openshift-dns...
2026-05-26T06:52:52.644978090Z Gathering data for ns/openshift-image-registry...
2026-05-26T06:52:52.841223513Z Waiting for performance profile collector pods to become ready: 3
2026-05-26T06:52:52.992310095Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-26T06:52:53.161852946Z Gathering data for ns/openshift-ingress-operator...
2026-05-26T06:52:53.484630142Z Gathering data for ns/openshift-ingress...
2026-05-26T06:52:53.822412121Z Gathering data for ns/openshift-ingress-canary...
2026-05-26T06:52:53.949530781Z Waiting for performance profile collector pods to become ready: 4
2026-05-26T06:52:54.253423585Z Gathering data for ns/openshift-insights...
2026-05-26T06:52:55.054028020Z Waiting for performance profile collector pods to become ready: 5
2026-05-26T06:52:56.175852522Z Waiting for performance profile collector pods to become ready: 6
2026-05-26T06:52:56.194444465Z Gathering data for ns/openshift-config...
2026-05-26T06:52:56.436194036Z Gathering data for ns/openshift-config-managed...
2026-05-26T06:52:56.819988251Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-26T06:52:57.064625805Z Gathering data for ns/openshift-kube-apiserver...
2026-05-26T06:52:57.296418342Z Waiting for performance profile collector pods to become ready: 7
2026-05-26T06:52:57.324123929Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-26T06:52:57.680529681Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-26T06:52:57.997989557Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-26T06:52:58.080789758Z Gathering data for ns/openshift-kube-scheduler...
2026-05-26T06:52:58.333213299Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-26T06:52:58.420164039Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-26T06:52:58.667061037Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-26T06:52:58.673516145Z Collecting performance related data for node ip-10-0-132-73.ec2.internal
2026-05-26T06:52:58.680206963Z Collecting performance related data for node ip-10-0-136-226.ec2.internal
2026-05-26T06:52:58.687809578Z Collecting performance related data for node ip-10-0-135-20.ec2.internal
2026-05-26T06:52:59.084864202Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-26T06:52:59.490241324Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-26T06:53:00.096052363Z Gathering data for ns/openshift-multus...
2026-05-26T06:53:01.701575092Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-26T06:53:03.003326964Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-26T06:53:03.246998635Z Gathering data for ns/openshift-host-network...
2026-05-26T06:53:03.573403405Z Gathering data for ns/openshift-network-diagnostics...
2026-05-26T06:53:04.126671586Z Gathering data for ns/openshift-network-node-identity...
2026-05-26T06:53:04.542531365Z Gathering data for ns/openshift-network-operator...
2026-05-26T06:53:04.945049424Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-26T06:53:05.185988578Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-26T06:53:05.658182321Z Gathering data for ns/openshift-apiserver-operator...
2026-05-26T06:53:05.988948797Z Gathering data for ns/openshift-apiserver...
2026-05-26T06:53:06.417217281Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-26T06:53:06.513781617Z Collecting kubelet logs for node ip-10-0-132-73.ec2.internal
2026-05-26T06:53:06.515492397Z Collecting kubelet logs for node ip-10-0-136-226.ec2.internal
2026-05-26T06:53:06.517656350Z Collecting kubelet logs for node ip-10-0-135-20.ec2.internal
2026-05-26T06:53:06.714635226Z Gathering data for ns/openshift-controller-manager...
2026-05-26T06:53:06.884800726Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-26T06:53:06.889968402Z INFO: Node performance data collection complete.
2026-05-26T06:53:07.009799039Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-26T06:53:07.705797389Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-26T06:53:07.980789557Z Gathering data for ns/openshift-service-ca-operator...
2026-05-26T06:53:08.008355489Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-26T06:53:08.275125644Z Gathering data for ns/openshift-service-ca...
2026-05-26T06:53:08.665512534Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-26T06:53:09.504233475Z Wrote inspect data to must-gather.
2026-05-26T06:53:09.504275918Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T06:53:09.504275918Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-26T06:53:09.624293585Z error: the server doesn't have a resource type "clusters"
2026-05-26T06:53:09.756718488Z Caches written to disk