2026-05-28T19:21:24.455864778Z [disk usage checker] Started 2026-05-28T19:21:24.459694890Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-05-28T19:21:24.696237299Z Gathering data for ns/openshift-cluster-version... 2026-05-28T19:21:24.851605569Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-28T19:21:24.945785212Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-28T19:21:25.046361013Z Gathering data for ns/default... 2026-05-28T19:21:25.426903205Z Gathering data for ns/openshift... 2026-05-28T19:21:25.923579049Z Gathering data for ns/kube-system... 2026-05-28T19:21:26.856776731Z Gathering data for ns/openshift-etcd... 2026-05-28T19:21:26.872259846Z Waiting on subprocesses to finish execution. 2026-05-28T19:21:26.886304431Z INFO: Gathering HAProxy config files 2026-05-28T19:21:26.886590802Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-05-28T19:21:26.893473285Z INFO: Gathering machine config daemon's old logs from all nodes 2026-05-28T19:21:26.894653413Z INFO: Collecting host service logs for crio 2026-05-28T19:21:26.894919038Z INFO: Collecting host service logs for kubelet 2026-05-28T19:21:26.895312096Z INFO: Collecting host service logs for rpm-ostreed 2026-05-28T19:21:26.895601995Z INFO: Collecting host service logs for ostree-finalize-staged 2026-05-28T19:21:26.895910321Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-05-28T19:21:26.896243595Z INFO: Collecting host service logs for machine-config-daemon-host 2026-05-28T19:21:26.896553143Z INFO: Collecting host service logs for NetworkManager 2026-05-28T19:21:26.896856214Z INFO: Collecting host service logs for openvswitch 2026-05-28T19:21:26.897156946Z INFO: Collecting host service logs for ovs-configuration 2026-05-28T19:21:26.897467507Z INFO: Collecting host service logs for ovsdb-server 2026-05-28T19:21:26.897776489Z INFO: Collecting host service logs for ovs-vswitchd 2026-05-28T19:21:26.898132146Z INFO: Waiting for worker host service log collection to complete ... 2026-05-28T19:21:26.901833606Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-05-28T19:21:26.904718449Z INFO: Waiting for node performance related collection to complete ... 2026-05-28T19:21:27.339390874Z INFO: Found 1 replicas - prometheus-k8s-0 2026-05-28T19:21:27.627479210Z error: the server doesn't have a resource type "performanceprofile" 2026-05-28T19:21:27.708472059Z No resources found in openshift-etcd namespace. 2026-05-28T19:21:28.030635094Z No resources found 2026-05-28T19:21:28.201166925Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-05-28T19:21:28.227620178Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-jx2bq 2026-05-28T19:21:28.227620178Z insights-runtime-extractor-6f9td 2026-05-28T19:21:28.227620178Z insights-runtime-extractor-c5t9b 2026-05-28T19:21:28.227620178Z insights-runtime-extractor-qkbpz 2026-05-28T19:21:28.378683731Z INFO: "sriov-network-operator" not detected. Skipping. 2026-05-28T19:21:28.385599945Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-05-28T19:21:28.385670815Z INFO: on-disk MachineConfig config collection complete. 2026-05-28T19:21:28.409736347Z INFO: Worker host service log collection to complete. 2026-05-28T19:21:28.487725757Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-05-28T19:21:28.590226398Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-05-28T19:21:28.711172588Z Wrote inspect data to must-gather. 2026-05-28T19:21:28.732239512Z INFO: "metallb-operator" not detected. Skipping. 2026-05-28T19:21:28.934050548Z Wrote inspect data to must-gather. 2026-05-28T19:21:28.934144565Z error: inspection completed with the errors occurred while gathering data: 2026-05-28T19:21:28.934144565Z namespaces "assisted-installer" not found 2026-05-28T19:21:28.965932482Z Wrote inspect data to must-gather. 2026-05-28T19:21:29.014837036Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-05-28T19:21:29.014837036Z See 'oc rsync -h' for help and examples 2026-05-28T19:21:29.157361015Z ERROR: No running kube-apiserver pods found 2026-05-28T19:21:29.241707921Z INFO: Waiting for HAProxy config collection to complete ... 2026-05-28T19:21:29.352266606Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-28T19:21:29.427297716Z No resources found 2026-05-28T19:21:29.511851563Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-05-28T19:21:29.537321334Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-05-28T19:21:29.617969426Z INFO: 'previous-logs' folder not found on ip-10-0-134-108.ec2.internal, skipping... 2026-05-28T19:21:29.658617236Z Wrote inspect data to must-gather. 2026-05-28T19:21:29.823683403Z Gathering data for ns/kuadrant-system... 2026-05-28T19:21:29.943657822Z Wrote inspect data to must-gather. 2026-05-28T19:21:29.994103718Z Wrote inspect data to must-gather. 2026-05-28T19:21:30.093657347Z INFO: Getting rules from prometheus-k8s-0 2026-05-28T19:21:30.113289182Z error: the server doesn't have a resource type "machineconfigs" 2026-05-28T19:21:30.155361893Z tar: Removing leading `/' from member names 2026-05-28T19:21:30.161319935Z INFO: HAProxy config collection complete. 2026-05-28T19:21:30.216683843Z INFO: 'previous-logs' folder not found on ip-10-0-135-133.ec2.internal, skipping... 2026-05-28T19:21:30.233370978Z Wrote inspect data to must-gather. 2026-05-28T19:21:30.287172269Z Wrote inspect data to must-gather. 2026-05-28T19:21:30.457292064Z Wrote inspect data to must-gather. 2026-05-28T19:21:30.614647537Z INFO: 'previous-logs' folder not found on ip-10-0-137-226.ec2.internal, skipping... 2026-05-28T19:21:30.614751046Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-05-28T19:21:30.614807047Z INFO: Machine Config Daemon termination log collection complete. 2026-05-28T19:21:30.656964406Z INFO: Getting status/config from prometheus-k8s-0 2026-05-28T19:21:30.705527490Z Wrote inspect data to must-gather. 2026-05-28T19:21:30.768399747Z Wrote inspect data to must-gather. 2026-05-28T19:21:31.037020718Z error: the server doesn't have a resource type "kubeletconfigs" 2026-05-28T19:21:31.080605569Z Wrote inspect data to must-gather. 2026-05-28T19:21:31.089796516Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-28T19:21:31.159284388Z INFO: Getting status/flags from prometheus-k8s-0 2026-05-28T19:21:31.350470435Z Wrote inspect data to must-gather. 2026-05-28T19:21:31.419047651Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-05-28T19:21:31.478420174Z INFO: INTERCONNECT MODE 2026-05-28T19:21:31.478515494Z INFO: Gathering ovn-kubernetes DBs 2026-05-28T19:21:31.623284330Z W0528 19:21:31.623244 1168 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-05-28T19:21:31.654033667Z Wrote inspect data to must-gather. 2026-05-28T19:21:31.684045323Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-28T19:21:31.704475403Z INFO: Gathering OVN_Northbound from ovnkube-node-9h2tl... 2026-05-28T19:21:31.704756215Z INFO: Gathering OVN_Northbound from ovnkube-node-drfnc... 2026-05-28T19:21:31.705000578Z INFO: Gathering OVN_Northbound from ovnkube-node-l99cm... 2026-05-28T19:21:31.705506674Z INFO: Gathering OVN_Southbound from ovnkube-node-9h2tl... 2026-05-28T19:21:31.705808251Z INFO: Gathering OVN_Southbound from ovnkube-node-drfnc... 2026-05-28T19:21:31.706067053Z INFO: Gathering OVN_Southbound from ovnkube-node-l99cm... 2026-05-28T19:21:31.707231148Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-05-28T19:21:32.015122461Z tar: Removing leading `/' from member names 2026-05-28T19:21:32.063383661Z Gathering data for ns/openshift-monitoring... 2026-05-28T19:21:32.075950254Z tar: Removing leading `/' from member names 2026-05-28T19:21:32.123024862Z tar: Removing leading `/' from member names 2026-05-28T19:21:32.184001474Z tar: Removing leading `/' from member names 2026-05-28T19:21:32.201910348Z tar: Removing leading `/' from member names 2026-05-28T19:21:32.207453113Z tar: Removing leading `/' from member names 2026-05-28T19:21:32.243306763Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-05-28T19:21:32.285914793Z Wrote inspect data to must-gather. 2026-05-28T19:21:32.666598906Z Wrote inspect data to must-gather. 2026-05-28T19:21:32.681225518Z INFO: Waiting for network log collection to complete ... 2026-05-28T19:21:32.681364486Z INFO: Waiting for ovnk database copies to complete ... 2026-05-28T19:21:32.681889816Z INFO: Copying ovnk databases complete. 2026-05-28T19:21:32.684048752Z 4.2M must-gather/network_logs/ovnk_database_store 2026-05-28T19:21:32.687369828Z ovnk_database_store/ 2026-05-28T19:21:32.687413137Z ovnk_database_store/ovnkube-node-drfnc_nbdb 2026-05-28T19:21:32.692168505Z INFO: Getting status from alertmanager-main-0 2026-05-28T19:21:32.736550526Z ovnk_database_store/ovnkube-node-drfnc_sbdb 2026-05-28T19:21:32.765996623Z ovnk_database_store/ovnkube-node-9h2tl_sbdb 2026-05-28T19:21:32.792300452Z ovnk_database_store/ovnkube-node-9h2tl_nbdb 2026-05-28T19:21:32.805499066Z ovnk_database_store/ovnkube-node-l99cm_sbdb 2026-05-28T19:21:32.831586112Z ovnk_database_store/ovnkube-node-l99cm_nbdb 2026-05-28T19:21:32.844811043Z INFO: Network log collection complete. 2026-05-28T19:21:32.975605556Z Wrote inspect data to must-gather. 2026-05-28T19:21:33.306543003Z Wrote inspect data to must-gather. 2026-05-28T19:21:33.583355770Z Wrote inspect data to must-gather. 2026-05-28T19:21:33.858300447Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-05-28T19:21:33.861161767Z INFO: Fallback to identify the container image from release info 2026-05-28T19:21:34.204242857Z Gathering data for ns/openshift-network-console... 2026-05-28T19:21:34.513306268Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda 2026-05-28T19:21:34.544535096Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-05-28T19:21:34.670054185Z daemonset.apps/perf-node-gather-daemonset created 2026-05-28T19:21:34.781402610Z Gathering data for ns/openshift-console-operator... 2026-05-28T19:21:34.831256237Z Waiting for performance profile collector pods to become ready: 1 2026-05-28T19:21:35.278574830Z Gathering data for ns/openshift-console... 2026-05-28T19:21:35.813670432Z Gathering data for ns/openshift-cluster-storage-operator... 2026-05-28T19:21:35.985718768Z Waiting for performance profile collector pods to become ready: 2 2026-05-28T19:21:36.271320979Z Gathering data for ns/openshift-dns-operator... 2026-05-28T19:21:36.485785551Z Wrote inspect data to must-gather. 2026-05-28T19:21:36.663318811Z Gathering data for ns/openshift-dns... 2026-05-28T19:21:37.117185007Z Waiting for performance profile collector pods to become ready: 3 2026-05-28T19:21:37.289316522Z Gathering data for ns/openshift-image-registry... 2026-05-28T19:21:37.744463111Z Gathering data for ns/openshift-ingress-operator... 2026-05-28T19:21:38.089715642Z Gathering data for ns/openshift-ingress... 2026-05-28T19:21:38.239994032Z Waiting for performance profile collector pods to become ready: 4 2026-05-28T19:21:38.902786053Z Gathering data for ns/openshift-ingress-canary... 2026-05-28T19:21:39.371825842Z Waiting for performance profile collector pods to become ready: 5 2026-05-28T19:21:39.382436295Z Gathering data for ns/openshift-insights... 2026-05-28T19:21:39.549956232Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-05-28T19:21:40.479203647Z Waiting for performance profile collector pods to become ready: 6 2026-05-28T19:21:41.429336446Z Gathering data for ns/opendatahub... 2026-05-28T19:21:41.591850702Z Waiting for performance profile collector pods to become ready: 7 2026-05-28T19:21:42.704672717Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-05-28T19:21:42.940910664Z Collecting performance related data for node ip-10-0-134-108.ec2.internal 2026-05-28T19:21:42.948012330Z Collecting performance related data for node ip-10-0-135-133.ec2.internal 2026-05-28T19:21:42.955961429Z Collecting performance related data for node ip-10-0-137-226.ec2.internal 2026-05-28T19:21:43.048640751Z Gathering data for ns/openshift-lws-operator... 2026-05-28T19:21:44.508941221Z Gathering data for ns/openshift-config... 2026-05-28T19:21:44.558318658Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-05-28T19:21:44.835505236Z Gathering data for ns/openshift-config-managed... 2026-05-28T19:21:45.354858215Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-05-28T19:21:45.702660913Z Gathering data for ns/openshift-kube-apiserver... 2026-05-28T19:21:46.053976511Z Gathering data for ns/openshift-kube-controller-manager... 2026-05-28T19:21:46.383824781Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-05-28T19:21:46.736108209Z Gathering data for ns/openshift-kube-scheduler... 2026-05-28T19:21:47.042643845Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-05-28T19:21:47.396100057Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-05-28T19:21:47.805247305Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-05-28T19:21:48.178369175Z Gathering data for ns/openshift-user-workload-monitoring... 2026-05-28T19:21:48.848261755Z Gathering data for ns/openshift-multus... 2026-05-28T19:21:49.563366761Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-05-28T19:21:50.374532253Z Gathering data for ns/openshift-ovn-kubernetes... 2026-05-28T19:21:51.108859356Z Collecting kubelet logs for node ip-10-0-134-108.ec2.internal 2026-05-28T19:21:51.111523809Z Collecting kubelet logs for node ip-10-0-135-133.ec2.internal 2026-05-28T19:21:51.114065537Z Collecting kubelet logs for node ip-10-0-137-226.ec2.internal 2026-05-28T19:21:51.484491573Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-05-28T19:21:51.489115676Z INFO: Node performance data collection complete. 2026-05-28T19:21:51.893256477Z Gathering data for ns/openshift-host-network... 2026-05-28T19:21:52.234597044Z Gathering data for ns/openshift-network-diagnostics... 2026-05-28T19:21:52.783504174Z Gathering data for ns/openshift-network-node-identity... 2026-05-28T19:21:53.255300834Z Gathering data for ns/openshift-network-operator... 2026-05-28T19:21:53.637929698Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-05-28T19:21:53.948260493Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-05-28T19:21:54.406295165Z Gathering data for ns/openshift-apiserver-operator... 2026-05-28T19:21:54.568195191Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-05-28T19:21:54.776594163Z Gathering data for ns/openshift-apiserver... 2026-05-28T19:21:55.203981997Z Gathering data for ns/openshift-controller-manager-operator... 2026-05-28T19:21:55.510239955Z Gathering data for ns/openshift-controller-manager... 2026-05-28T19:21:55.841673266Z Gathering data for ns/openshift-cluster-samples-operator... 2026-05-28T19:21:56.490478702Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-05-28T19:21:56.832981684Z Gathering data for ns/openshift-service-ca-operator... 2026-05-28T19:21:57.162370993Z Gathering data for ns/openshift-service-ca... 2026-05-28T19:21:57.568382978Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-05-28T19:21:58.573827373Z Wrote inspect data to must-gather. 2026-05-28T19:21:58.573868025Z error: inspection completed with the errors occurred while gathering data: 2026-05-28T19:21:58.573868025Z skipping gathering secrets/support due to error: secrets "support" not found 2026-05-28T19:21:58.707008941Z error: the server doesn't have a resource type "clusters" 2026-05-28T19:21:58.932103827Z Caches written to disk