2026-04-16T15:26:04.765697497Z [disk usage checker] Started 2026-04-16T15:26:04.800372426Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-04-16T15:26:05.152996069Z Gathering data for ns/openshift-cluster-version... 2026-04-16T15:26:05.302509520Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T15:26:05.393720623Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T15:26:05.490839826Z Gathering data for ns/default... 2026-04-16T15:26:05.819496597Z Gathering data for ns/openshift... 2026-04-16T15:26:06.271812274Z Gathering data for ns/kube-system... 2026-04-16T15:26:06.924154533Z Gathering data for ns/openshift-etcd... 2026-04-16T15:26:07.265832351Z Wrote inspect data to must-gather. 2026-04-16T15:26:07.265952947Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T15:26:07.265952947Z namespaces "assisted-installer" not found 2026-04-16T15:26:07.588454053Z Waiting on subprocesses to finish execution. 2026-04-16T15:26:07.597139026Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-16T15:26:07.598360646Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-16T15:26:07.601778704Z INFO: Gathering HAProxy config files 2026-04-16T15:26:07.607436029Z INFO: Collecting host service logs for crio 2026-04-16T15:26:07.607744129Z INFO: Collecting host service logs for kubelet 2026-04-16T15:26:07.608070489Z INFO: Collecting host service logs for rpm-ostreed 2026-04-16T15:26:07.608359883Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-16T15:26:07.608407533Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-16T15:26:07.608662046Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-16T15:26:07.608977838Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-16T15:26:07.609277211Z INFO: Collecting host service logs for NetworkManager 2026-04-16T15:26:07.609570840Z INFO: Collecting host service logs for openvswitch 2026-04-16T15:26:07.609873031Z INFO: Collecting host service logs for ovs-configuration 2026-04-16T15:26:07.610264259Z INFO: Collecting host service logs for ovsdb-server 2026-04-16T15:26:07.610579163Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-16T15:26:07.610983092Z INFO: Waiting for worker host service log collection to complete ... 2026-04-16T15:26:07.860803893Z INFO: Waiting for node performance related collection to complete ... 2026-04-16T15:26:08.239988736Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-tx72c 2026-04-16T15:26:08.240046475Z insights-runtime-extractor-8j958 2026-04-16T15:26:08.240067069Z insights-runtime-extractor-grl9h 2026-04-16T15:26:08.240086703Z insights-runtime-extractor-ktl5m 2026-04-16T15:26:08.315832775Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-16T15:26:08.613235513Z No resources found 2026-04-16T15:26:08.615350330Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-16T15:26:08.676940923Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-16T15:26:08.676985909Z INFO: on-disk MachineConfig config collection complete. 2026-04-16T15:26:08.692183859Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-16T15:26:08.787999911Z INFO: "metallb-operator" not detected. Skipping. 2026-04-16T15:26:08.792754290Z No resources found in openshift-etcd namespace. 2026-04-16T15:26:08.924492134Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-16T15:26:08.949277545Z Wrote inspect data to must-gather. 2026-04-16T15:26:08.973152423Z INFO: Worker host service log collection to complete. 2026-04-16T15:26:09.041168243Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-16T15:26:09.041168243Z See 'oc rsync -h' for help and examples 2026-04-16T15:26:09.204210180Z error: the server doesn't have a resource type "performanceprofile" 2026-04-16T15:26:09.552829116Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-16T15:26:09.636570098Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-16T15:26:09.932106010Z No resources found 2026-04-16T15:26:09.939881741Z INFO: 'previous-logs' folder not found on ip-10-0-137-160.ec2.internal, skipping... 2026-04-16T15:26:09.941298900Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-16T15:26:09.964503798Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-04-16T15:26:10.109180052Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T15:26:10.208525037Z INFO: HAProxy config collection complete. 2026-04-16T15:26:10.319742998Z Wrote inspect data to must-gather. 2026-04-16T15:26:10.368933466Z ERROR: No running kube-apiserver pods found 2026-04-16T15:26:10.384264196Z Wrote inspect data to must-gather. 2026-04-16T15:26:10.407098037Z Wrote inspect data to must-gather. 2026-04-16T15:26:10.480231080Z Wrote inspect data to must-gather. 2026-04-16T15:26:10.581003842Z INFO: 'previous-logs' folder not found on ip-10-0-139-47.ec2.internal, skipping... 2026-04-16T15:26:10.596006888Z Gathering data for ns/kuadrant-system... 2026-04-16T15:26:10.603769168Z Wrote inspect data to must-gather. 2026-04-16T15:26:10.733139044Z INFO: Getting rules from prometheus-k8s-0 2026-04-16T15:26:10.764709088Z Wrote inspect data to must-gather. 2026-04-16T15:26:10.892378173Z error: the server doesn't have a resource type "machineconfigs" 2026-04-16T15:26:10.993768521Z INFO: 'previous-logs' folder not found on ip-10-0-140-83.ec2.internal, skipping... 2026-04-16T15:26:10.993884907Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-16T15:26:10.993967802Z INFO: Machine Config Daemon termination log collection complete. 2026-04-16T15:26:11.125091480Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T15:26:11.165862609Z Wrote inspect data to must-gather. 2026-04-16T15:26:11.232398110Z Wrote inspect data to must-gather. 2026-04-16T15:26:11.324345468Z INFO: Getting status/config from prometheus-k8s-0 2026-04-16T15:26:11.424665893Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T15:26:11.602364163Z Wrote inspect data to must-gather. 2026-04-16T15:26:11.628797260Z Wrote inspect data to must-gather. 2026-04-16T15:26:11.635044546Z Gathering data for ns/openshift-monitoring... 2026-04-16T15:26:11.842189258Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-16T15:26:11.867952660Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-16T15:26:12.005268799Z INFO: INTERCONNECT MODE 2026-04-16T15:26:12.005268799Z INFO: Gathering ovn-kubernetes DBs 2026-04-16T15:26:12.099861486Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-16T15:26:12.208251673Z Wrote inspect data to must-gather. 2026-04-16T15:26:12.211881640Z INFO: Gathering OVN_Northbound from ovnkube-node-bff4l... 2026-04-16T15:26:12.212237843Z INFO: Gathering OVN_Northbound from ovnkube-node-czzzx... 2026-04-16T15:26:12.212480997Z INFO: Gathering OVN_Northbound from ovnkube-node-wrxcw... 2026-04-16T15:26:12.212749614Z INFO: Gathering OVN_Southbound from ovnkube-node-bff4l... 2026-04-16T15:26:12.213018244Z INFO: Gathering OVN_Southbound from ovnkube-node-czzzx... 2026-04-16T15:26:12.213299085Z INFO: Gathering OVN_Southbound from ovnkube-node-wrxcw... 2026-04-16T15:26:12.439338877Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-16T15:26:12.541161987Z tar: Removing leading `/' from member names 2026-04-16T15:26:12.551237060Z tar: Removing leading `/' from member names 2026-04-16T15:26:12.573766334Z tar: Removing leading `/' from member names 2026-04-16T15:26:12.607088943Z tar: Removing leading `/' from member names 2026-04-16T15:26:12.648424934Z tar: Removing leading `/' from member names 2026-04-16T15:26:12.674989070Z tar: Removing leading `/' from member names 2026-04-16T15:26:12.730553567Z Wrote inspect data to must-gather. 2026-04-16T15:26:12.828991669Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-16T15:26:13.130263323Z Wrote inspect data to must-gather. 2026-04-16T15:26:13.361929823Z INFO: Getting status from alertmanager-main-0 2026-04-16T15:26:13.392555786Z INFO: Waiting for network log collection to complete ... 2026-04-16T15:26:13.392695864Z INFO: Waiting for ovnk database copies to complete ... 2026-04-16T15:26:13.393158825Z INFO: Copying ovnk databases complete. 2026-04-16T15:26:13.395463328Z 4.9M must-gather/network_logs/ovnk_database_store 2026-04-16T15:26:13.403131099Z ovnk_database_store/ 2026-04-16T15:26:13.403791683Z ovnk_database_store/ovnkube-node-czzzx_sbdb 2026-04-16T15:26:13.441226391Z ovnk_database_store/ovnkube-node-bff4l_sbdb 2026-04-16T15:26:13.472912122Z Wrote inspect data to must-gather. 2026-04-16T15:26:13.475514112Z ovnk_database_store/ovnkube-node-bff4l_nbdb 2026-04-16T15:26:13.489662746Z ovnk_database_store/ovnkube-node-czzzx_nbdb 2026-04-16T15:26:13.501219884Z ovnk_database_store/ovnkube-node-wrxcw_nbdb 2026-04-16T15:26:13.513581110Z ovnk_database_store/ovnkube-node-wrxcw_sbdb 2026-04-16T15:26:13.543837052Z INFO: Network log collection complete. 2026-04-16T15:26:13.740323202Z W0416 15:26:13.740268 1400 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-04-16T15:26:13.750013266Z Wrote inspect data to must-gather. 2026-04-16T15:26:14.065809546Z Gathering data for ns/openshift-network-console... 2026-04-16T15:26:14.123292157Z Wrote inspect data to must-gather. 2026-04-16T15:26:14.507871170Z Wrote inspect data to must-gather. 2026-04-16T15:26:14.593178556Z Gathering data for ns/openshift-console-operator... 2026-04-16T15:26:14.827964379Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-16T15:26:14.833126866Z INFO: Fallback to identify the container image from release info 2026-04-16T15:26:14.978261727Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-04-16T15:26:15.071729246Z Gathering data for ns/openshift-console... 2026-04-16T15:26:15.270277737Z Wrote inspect data to must-gather. 2026-04-16T15:26:15.336473805Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9 2026-04-16T15:26:15.472975987Z daemonset.apps/perf-node-gather-daemonset created 2026-04-16T15:26:15.547939587Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-16T15:26:15.624744485Z Waiting for performance profile collector pods to become ready: 1 2026-04-16T15:26:15.913118663Z Gathering data for ns/openshift-dns-operator... 2026-04-16T15:26:16.235506103Z Gathering data for ns/openshift-dns... 2026-04-16T15:26:16.771796404Z Waiting for performance profile collector pods to become ready: 2 2026-04-16T15:26:16.861981630Z Gathering data for ns/openshift-image-registry... 2026-04-16T15:26:17.314481173Z Gathering data for ns/openshift-ingress-operator... 2026-04-16T15:26:17.654239499Z Gathering data for ns/openshift-ingress... 2026-04-16T15:26:17.917606883Z Waiting for performance profile collector pods to become ready: 3 2026-04-16T15:26:18.185366361Z Gathering data for ns/openshift-ingress-canary... 2026-04-16T15:26:18.599742145Z Gathering data for ns/openshift-insights... 2026-04-16T15:26:19.048749083Z Waiting for performance profile collector pods to become ready: 4 2026-04-16T15:26:19.983224682Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-04-16T15:26:20.169672907Z Waiting for performance profile collector pods to become ready: 5 2026-04-16T15:26:21.269716397Z Gathering data for ns/openshift-lws-operator... 2026-04-16T15:26:21.298866618Z Waiting for performance profile collector pods to become ready: 6 2026-04-16T15:26:21.804501188Z Gathering data for ns/kserve... 2026-04-16T15:26:22.441204215Z Waiting for performance profile collector pods to become ready: 7 2026-04-16T15:26:23.585788498Z Waiting for performance profile collector pods to become ready: 8 2026-04-16T15:26:23.682057814Z Gathering data for ns/openshift-config... 2026-04-16T15:26:23.958153973Z Gathering data for ns/openshift-config-managed... 2026-04-16T15:26:24.399961485Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-16T15:26:24.675738799Z Gathering data for ns/openshift-kube-apiserver... 2026-04-16T15:26:24.739811123Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-16T15:26:24.989522804Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-04-16T15:26:24.994848196Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-16T15:26:25.062078403Z Collecting performance related data for node ip-10-0-139-47.ec2.internal 2026-04-16T15:26:25.072549592Z Collecting performance related data for node ip-10-0-137-160.ec2.internal 2026-04-16T15:26:25.081182028Z Collecting performance related data for node ip-10-0-140-83.ec2.internal 2026-04-16T15:26:25.361889992Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-16T15:26:25.799782109Z Gathering data for ns/openshift-kube-scheduler... 2026-04-16T15:26:26.158497303Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-16T15:26:26.610806105Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-16T15:26:27.009186790Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-16T15:26:27.324418288Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-16T15:26:27.962641456Z Gathering data for ns/openshift-multus... 2026-04-16T15:26:29.325508371Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-16T15:26:29.996151945Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-04-16T15:26:30.873267174Z Gathering data for ns/openshift-host-network... 2026-04-16T15:26:31.247888822Z Gathering data for ns/openshift-network-diagnostics... 2026-04-16T15:26:31.884791707Z Gathering data for ns/openshift-network-node-identity... 2026-04-16T15:26:32.349778770Z Gathering data for ns/openshift-network-operator... 2026-04-16T15:26:32.766590318Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-16T15:26:33.109966760Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-16T15:26:33.528564048Z Collecting kubelet logs for node ip-10-0-139-47.ec2.internal 2026-04-16T15:26:33.530961847Z Collecting kubelet logs for node ip-10-0-137-160.ec2.internal 2026-04-16T15:26:33.533435073Z Collecting kubelet logs for node ip-10-0-140-83.ec2.internal 2026-04-16T15:26:33.613998605Z Gathering data for ns/openshift-apiserver-operator... 2026-04-16T15:26:33.990058588Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-16T15:26:33.997650017Z INFO: Node performance data collection complete. 2026-04-16T15:26:34.023880258Z Gathering data for ns/openshift-apiserver... 2026-04-16T15:26:34.478087008Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-16T15:26:34.747293373Z Gathering data for ns/openshift-controller-manager... 2026-04-16T15:26:35.002518706Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-04-16T15:26:35.056040152Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-16T15:26:35.687705895Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-16T15:26:35.977202830Z Gathering data for ns/openshift-service-ca-operator... 2026-04-16T15:26:36.289559096Z Gathering data for ns/openshift-service-ca... 2026-04-16T15:26:36.653055082Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-16T15:26:37.559620117Z Wrote inspect data to must-gather. 2026-04-16T15:26:37.559665231Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T15:26:37.559665231Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-16T15:26:37.700300385Z error: the server doesn't have a resource type "clusters" 2026-04-16T15:26:37.870766176Z Caches written to disk