2026-05-26T17:35:06.239168501Z [disk usage checker] Started 2026-05-26T17:35:06.243275159Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 2026-05-26T17:35:06.431052318Z Gathering data for ns/openshift-cluster-version... 2026-05-26T17:35:06.563448862Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-26T17:35:06.634042422Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-26T17:35:06.707254158Z Gathering data for ns/default... 2026-05-26T17:35:06.962083925Z Gathering data for ns/openshift... 2026-05-26T17:35:07.319317195Z Gathering data for ns/kube-system... 2026-05-26T17:35:07.933092072Z Gathering data for ns/openshift-etcd... 2026-05-26T17:35:08.196227574Z Wrote inspect data to must-gather. 2026-05-26T17:35:08.196261956Z error: inspection completed with the errors occurred while gathering data: 2026-05-26T17:35:08.196261956Z namespaces "assisted-installer" not found 2026-05-26T17:35:08.467256728Z Waiting on subprocesses to finish execution. 2026-05-26T17:35:08.479352106Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-05-26T17:35:08.482368808Z INFO: Gathering HAProxy config files 2026-05-26T17:35:08.483977003Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-05-26T17:35:08.484115171Z INFO: Gathering machine config daemon's old logs from all nodes 2026-05-26T17:35:08.490960323Z INFO: Collecting host service logs for crio 2026-05-26T17:35:08.491357849Z INFO: Collecting host service logs for kubelet 2026-05-26T17:35:08.491982868Z INFO: Collecting host service logs for rpm-ostreed 2026-05-26T17:35:08.492320840Z INFO: Collecting host service logs for ostree-finalize-staged 2026-05-26T17:35:08.492649422Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-05-26T17:35:08.493073913Z INFO: Collecting host service logs for machine-config-daemon-host 2026-05-26T17:35:08.493435258Z INFO: Collecting host service logs for NetworkManager 2026-05-26T17:35:08.493763463Z INFO: Collecting host service logs for openvswitch 2026-05-26T17:35:08.494680070Z INFO: Collecting host service logs for ovs-configuration 2026-05-26T17:35:08.495062381Z INFO: Collecting host service logs for ovsdb-server 2026-05-26T17:35:08.495694237Z INFO: Collecting host service logs for ovs-vswitchd 2026-05-26T17:35:08.496118898Z INFO: Waiting for worker host service log collection to complete ... 2026-05-26T17:35:08.842461943Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-05-26T17:35:08.902801005Z INFO: Waiting for node performance related collection to complete ... 2026-05-26T17:35:09.329248632Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-w47qs 2026-05-26T17:35:09.329248632Z insights-runtime-extractor-89mcs 2026-05-26T17:35:09.329248632Z insights-runtime-extractor-qfl7k 2026-05-26T17:35:09.329248632Z insights-runtime-extractor-tk2sx 2026-05-26T17:35:09.444626865Z INFO: Found 1 replicas - prometheus-k8s-0 2026-05-26T17:35:09.609906170Z No resources found in openshift-etcd namespace. 2026-05-26T17:35:09.610635812Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-05-26T17:35:09.710778249Z INFO: "metallb-operator" not detected. Skipping. 2026-05-26T17:35:09.756976766Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-05-26T17:35:09.757072803Z INFO: on-disk MachineConfig config collection complete. 2026-05-26T17:35:09.834144214Z No resources found 2026-05-26T17:35:09.854317843Z Wrote inspect data to must-gather. 2026-05-26T17:35:09.954992843Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-05-26T17:35:09.954992843Z See 'oc rsync -h' for help and examples 2026-05-26T17:35:09.977928876Z INFO: "sriov-network-operator" not detected. Skipping. 2026-05-26T17:35:10.006777701Z INFO: Worker host service log collection to complete. 2026-05-26T17:35:10.068416230Z INFO: 'previous-logs' folder not found on ip-10-0-129-109.ec2.internal, skipping... 2026-05-26T17:35:10.108117016Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-05-26T17:35:10.228823426Z ERROR: No running kube-apiserver pods found 2026-05-26T17:35:10.257595521Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-26T17:35:10.346941447Z Wrote inspect data to must-gather. 2026-05-26T17:35:10.356304122Z Wrote inspect data to must-gather. 2026-05-26T17:35:10.357665057Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-05-26T17:35:10.434212285Z No resources found 2026-05-26T17:35:10.471535782Z INFO: Waiting for HAProxy config collection to complete ... 2026-05-26T17:35:10.501957037Z error: the server doesn't have a resource type "performanceprofile" 2026-05-26T17:35:10.516985625Z Wrote inspect data to must-gather. 2026-05-26T17:35:10.582100470Z Wrote inspect data to must-gather. 2026-05-26T17:35:10.612771830Z Wrote inspect data to must-gather. 2026-05-26T17:35:10.759263568Z INFO: 'previous-logs' folder not found on ip-10-0-136-115.ec2.internal, skipping... 2026-05-26T17:35:10.805968429Z Wrote inspect data to must-gather. 2026-05-26T17:35:10.806718001Z tar: Removing leading `/' from member names 2026-05-26T17:35:10.806908008Z Wrote inspect data to must-gather. 2026-05-26T17:35:10.815172993Z INFO: HAProxy config collection complete. 2026-05-26T17:35:10.848958445Z INFO: Getting rules from prometheus-k8s-0 2026-05-26T17:35:10.859986192Z Gathering data for ns/openshift-monitoring... 2026-05-26T17:35:10.971185638Z error: the server doesn't have a resource type "machineconfigs" 2026-05-26T17:35:11.042735870Z INFO: 'previous-logs' folder not found on ip-10-0-142-176.ec2.internal, skipping... 2026-05-26T17:35:11.042775030Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-05-26T17:35:11.042795746Z INFO: Machine Config Daemon termination log collection complete. 2026-05-26T17:35:11.055661025Z Wrote inspect data to must-gather. 2026-05-26T17:35:11.188120920Z Wrote inspect data to must-gather. 2026-05-26T17:35:11.252520838Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 2026-05-26T17:35:11.256314253Z INFO: Getting status/config from prometheus-k8s-0 2026-05-26T17:35:11.284755536Z Wrote inspect data to must-gather. 2026-05-26T17:35:11.403471527Z Wrote inspect data to must-gather. 2026-05-26T17:35:11.537742708Z error: the server doesn't have a resource type "kubeletconfigs" 2026-05-26T17:35:11.548285842Z INFO: INTERCONNECT MODE 2026-05-26T17:35:11.548311962Z INFO: Gathering ovn-kubernetes DBs 2026-05-26T17:35:11.612324264Z INFO: Getting status/flags from prometheus-k8s-0 2026-05-26T17:35:11.725051107Z INFO: Gathering OVN_Northbound from ovnkube-node-p2zsh... 2026-05-26T17:35:11.725305732Z INFO: Gathering OVN_Northbound from ovnkube-node-r56x2... 2026-05-26T17:35:11.725609676Z INFO: Gathering OVN_Northbound from ovnkube-node-tktl4... 2026-05-26T17:35:11.725894506Z INFO: Gathering OVN_Southbound from ovnkube-node-p2zsh... 2026-05-26T17:35:11.726165340Z INFO: Gathering OVN_Southbound from ovnkube-node-r56x2... 2026-05-26T17:35:11.726450117Z INFO: Gathering OVN_Southbound from ovnkube-node-tktl4... 2026-05-26T17:35:11.741906955Z Wrote inspect data to must-gather. 2026-05-26T17:35:11.847474411Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-05-26T17:35:12.017732507Z tar: Removing leading `/' from member names 2026-05-26T17:35:12.017801616Z tar: Removing leading `/' from member names 2026-05-26T17:35:12.017850093Z tar: Removing leading `/' from member names 2026-05-26T17:35:12.024195816Z tar: Removing leading `/' from member names 2026-05-26T17:35:12.040721538Z tar: Removing leading `/' from member names 2026-05-26T17:35:12.041965201Z tar: Removing leading `/' from member names 2026-05-26T17:35:12.083684256Z Wrote inspect data to must-gather. 2026-05-26T17:35:12.156733232Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-05-26T17:35:12.158891680Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-26T17:35:12.311840120Z Wrote inspect data to must-gather. 2026-05-26T17:35:12.419099730Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-05-26T17:35:12.459282623Z INFO: Waiting for network log collection to complete ... 2026-05-26T17:35:12.459406314Z INFO: Waiting for ovnk database copies to complete ... 2026-05-26T17:35:12.459935706Z INFO: Copying ovnk databases complete. 2026-05-26T17:35:12.461860400Z 3.7M must-gather/network_logs/ovnk_database_store 2026-05-26T17:35:12.464515829Z ovnk_database_store/ 2026-05-26T17:35:12.464515829Z ovnk_database_store/ovnkube-node-r56x2_sbdb 2026-05-26T17:35:12.526573947Z ovnk_database_store/ovnkube-node-p2zsh_sbdb 2026-05-26T17:35:12.529177453Z Wrote inspect data to must-gather. 2026-05-26T17:35:12.548050853Z ovnk_database_store/ovnkube-node-p2zsh_nbdb 2026-05-26T17:35:12.557003990Z ovnk_database_store/ovnkube-node-r56x2_nbdb 2026-05-26T17:35:12.568056546Z ovnk_database_store/ovnkube-node-tktl4_sbdb 2026-05-26T17:35:12.588436899Z ovnk_database_store/ovnkube-node-tktl4_nbdb 2026-05-26T17:35:12.599339165Z INFO: Network log collection complete. 2026-05-26T17:35:12.711109359Z W0526 17:35:12.711048 1370 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-05-26T17:35:12.715744805Z Wrote inspect data to must-gather. 2026-05-26T17:35:12.739121546Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-26T17:35:12.766326595Z INFO: Getting status from alertmanager-main-0 2026-05-26T17:35:12.900332427Z Gathering data for ns/openshift-network-console... 2026-05-26T17:35:13.004868051Z Wrote inspect data to must-gather. 2026-05-26T17:35:13.267891956Z Wrote inspect data to must-gather. 2026-05-26T17:35:13.288541031Z Gathering data for ns/openshift-console-operator... 2026-05-26T17:35:13.502842184Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-05-26T17:35:13.505443651Z INFO: Fallback to identify the container image from release info 2026-05-26T17:35:13.621510530Z Gathering data for ns/openshift-console... 2026-05-26T17:35:13.981392716Z Gathering data for ns/openshift-cluster-storage-operator... 2026-05-26T17:35:14.249319707Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda 2026-05-26T17:35:14.279106989Z Gathering data for ns/openshift-dns-operator... 2026-05-26T17:35:14.374771111Z daemonset.apps/perf-node-gather-daemonset created 2026-05-26T17:35:14.501878404Z Waiting for performance profile collector pods to become ready: 1 2026-05-26T17:35:14.548304898Z Gathering data for ns/openshift-dns... 2026-05-26T17:35:15.067094242Z Gathering data for ns/openshift-image-registry... 2026-05-26T17:35:15.457208489Z Gathering data for ns/openshift-ingress-operator... 2026-05-26T17:35:15.618778675Z Waiting for performance profile collector pods to become ready: 2 2026-05-26T17:35:15.728533056Z Gathering data for ns/openshift-ingress... 2026-05-26T17:35:16.001333597Z Gathering data for ns/openshift-ingress-canary... 2026-05-26T17:35:16.259154763Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 2026-05-26T17:35:16.305926748Z Gathering data for ns/openshift-insights... 2026-05-26T17:35:16.743209675Z Waiting for performance profile collector pods to become ready: 3 2026-05-26T17:35:17.853712944Z Waiting for performance profile collector pods to become ready: 4 2026-05-26T17:35:18.198304608Z Gathering data for ns/openshift-config... 2026-05-26T17:35:18.414626417Z Gathering data for ns/openshift-config-managed... 2026-05-26T17:35:18.765509466Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-05-26T17:35:18.982055262Z Waiting for performance profile collector pods to become ready: 5 2026-05-26T17:35:18.996903956Z Gathering data for ns/openshift-kube-apiserver... 2026-05-26T17:35:19.209485715Z Gathering data for ns/openshift-kube-controller-manager... 2026-05-26T17:35:19.441530366Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-05-26T17:35:19.675566886Z Gathering data for ns/openshift-kube-scheduler... 2026-05-26T17:35:19.935326942Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-05-26T17:35:20.103219491Z Waiting for performance profile collector pods to become ready: 6 2026-05-26T17:35:20.187119374Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-05-26T17:35:20.447102577Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-05-26T17:35:20.712039202Z Gathering data for ns/openshift-user-workload-monitoring... 2026-05-26T17:35:21.153532311Z Gathering data for ns/openshift-multus... 2026-05-26T17:35:21.217905529Z Waiting for performance profile collector pods to become ready: 7 2026-05-26T17:35:21.263709841Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 2026-05-26T17:35:22.345569126Z Gathering data for ns/openshift-ovn-kubernetes... 2026-05-26T17:35:22.346519993Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-05-26T17:35:22.560478069Z Collecting performance related data for node ip-10-0-129-109.ec2.internal 2026-05-26T17:35:22.567346640Z Collecting performance related data for node ip-10-0-142-176.ec2.internal 2026-05-26T17:35:22.575730786Z Collecting performance related data for node ip-10-0-136-115.ec2.internal 2026-05-26T17:35:23.897371953Z Gathering data for ns/openshift-host-network... 2026-05-26T17:35:24.191293178Z Gathering data for ns/openshift-network-diagnostics... 2026-05-26T17:35:24.688375212Z Gathering data for ns/openshift-network-node-identity... 2026-05-26T17:35:25.112979512Z Gathering data for ns/openshift-network-operator... 2026-05-26T17:35:25.470784841Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-05-26T17:35:25.696732731Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-05-26T17:35:26.084977413Z Gathering data for ns/openshift-apiserver-operator... 2026-05-26T17:35:26.269618329Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 2026-05-26T17:35:26.342101598Z Gathering data for ns/openshift-apiserver... 2026-05-26T17:35:26.684365988Z Gathering data for ns/openshift-controller-manager-operator... 2026-05-26T17:35:26.906983037Z Gathering data for ns/openshift-controller-manager... 2026-05-26T17:35:27.155852924Z Gathering data for ns/openshift-cluster-samples-operator... 2026-05-26T17:35:27.803367678Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-05-26T17:35:28.055760983Z Gathering data for ns/openshift-service-ca-operator... 2026-05-26T17:35:28.320845515Z Gathering data for ns/openshift-service-ca... 2026-05-26T17:35:28.632048639Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-05-26T17:35:29.463301023Z Wrote inspect data to must-gather. 2026-05-26T17:35:29.463347088Z error: inspection completed with the errors occurred while gathering data: 2026-05-26T17:35:29.463347088Z skipping gathering secrets/support due to error: secrets "support" not found 2026-05-26T17:35:30.766212537Z Collecting kubelet logs for node ip-10-0-129-109.ec2.internal 2026-05-26T17:35:30.767954527Z Collecting kubelet logs for node ip-10-0-142-176.ec2.internal 2026-05-26T17:35:30.769731960Z Collecting kubelet logs for node ip-10-0-136-115.ec2.internal 2026-05-26T17:35:31.081452473Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-05-26T17:35:31.085916213Z INFO: Node performance data collection complete. 2026-05-26T17:35:31.215120495Z error: the server doesn't have a resource type "clusters" 2026-05-26T17:35:31.274533466Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 2026-05-26T17:35:31.387716279Z Caches written to disk