2026-05-29T03:22:13.927955337Z [disk usage checker] Started 2026-05-29T03:22:13.931468192Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-05-29T03:22:14.130020787Z Gathering data for ns/openshift-cluster-version... 2026-05-29T03:22:14.272854663Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-29T03:22:14.357461139Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-29T03:22:14.442509152Z Gathering data for ns/default... 2026-05-29T03:22:14.729714728Z Gathering data for ns/openshift... 2026-05-29T03:22:15.165396509Z Gathering data for ns/kube-system... 2026-05-29T03:22:15.927793617Z Gathering data for ns/openshift-etcd... 2026-05-29T03:22:16.164602867Z Waiting on subprocesses to finish execution. 2026-05-29T03:22:16.170705623Z INFO: Gathering HAProxy config files 2026-05-29T03:22:16.177741085Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-05-29T03:22:16.181100734Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-05-29T03:22:16.182780322Z INFO: Collecting host service logs for crio 2026-05-29T03:22:16.183062575Z INFO: Collecting host service logs for kubelet 2026-05-29T03:22:16.183427602Z INFO: Collecting host service logs for rpm-ostreed 2026-05-29T03:22:16.183712511Z INFO: Collecting host service logs for ostree-finalize-staged 2026-05-29T03:22:16.184004741Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-05-29T03:22:16.184323889Z INFO: Collecting host service logs for machine-config-daemon-host 2026-05-29T03:22:16.184620479Z INFO: Collecting host service logs for NetworkManager 2026-05-29T03:22:16.184912911Z INFO: Collecting host service logs for openvswitch 2026-05-29T03:22:16.184912911Z INFO: Gathering machine config daemon's old logs from all nodes 2026-05-29T03:22:16.185206030Z INFO: Collecting host service logs for ovs-configuration 2026-05-29T03:22:16.185534162Z INFO: Collecting host service logs for ovsdb-server 2026-05-29T03:22:16.185826767Z INFO: Collecting host service logs for ovs-vswitchd 2026-05-29T03:22:16.186218545Z INFO: Waiting for worker host service log collection to complete ... 2026-05-29T03:22:16.212612674Z INFO: Waiting for node performance related collection to complete ... 2026-05-29T03:22:16.305211404Z Wrote inspect data to must-gather. 2026-05-29T03:22:16.305335486Z error: inspection completed with the errors occurred while gathering data: 2026-05-29T03:22:16.305335486Z namespaces "assisted-installer" not found 2026-05-29T03:22:16.730414185Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-05-29T03:22:16.730504529Z INFO: on-disk MachineConfig config collection complete. 2026-05-29T03:22:16.900239525Z No resources found 2026-05-29T03:22:17.058368619Z INFO: "sriov-network-operator" not detected. Skipping. 2026-05-29T03:22:17.406337016Z error: the server doesn't have a resource type "performanceprofile" 2026-05-29T03:22:17.420968120Z No resources found 2026-05-29T03:22:17.424066979Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-05-29T03:22:17.433741756Z INFO: "metallb-operator" not detected. Skipping. 2026-05-29T03:22:17.471369712Z Wrote inspect data to must-gather. 2026-05-29T03:22:17.647167113Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-29T03:22:17.740012874Z Wrote inspect data to must-gather. 2026-05-29T03:22:17.767626797Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-rrqp6 2026-05-29T03:22:17.767676243Z insights-runtime-extractor-9thz9 2026-05-29T03:22:17.767696373Z insights-runtime-extractor-jlzfg 2026-05-29T03:22:17.767715240Z insights-runtime-extractor-nhzfp 2026-05-29T03:22:17.820064614Z Wrote inspect data to must-gather. 2026-05-29T03:22:17.839664111Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-05-29T03:22:17.886359949Z Wrote inspect data to must-gather. 2026-05-29T03:22:17.936394518Z INFO: 'previous-logs' folder not found on ip-10-0-137-122.ec2.internal, skipping... 2026-05-29T03:22:17.936912288Z No resources found in openshift-etcd namespace. 2026-05-29T03:22:18.058447108Z INFO: Found 1 replicas - prometheus-k8s-0 2026-05-29T03:22:18.116104773Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-05-29T03:22:18.137772612Z Wrote inspect data to must-gather. 2026-05-29T03:22:18.152059267Z Wrote inspect data to must-gather. 2026-05-29T03:22:18.152088056Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-05-29T03:22:18.152088056Z See 'oc rsync -h' for help and examples 2026-05-29T03:22:18.219480130Z INFO: Worker host service log collection to complete. 2026-05-29T03:22:18.272099738Z INFO: Waiting for HAProxy config collection to complete ... 2026-05-29T03:22:18.374856164Z Wrote inspect data to must-gather. 2026-05-29T03:22:18.383801606Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-05-29T03:22:18.459176333Z Wrote inspect data to must-gather. 2026-05-29T03:22:18.464795497Z ERROR: No running kube-apiserver pods found 2026-05-29T03:22:18.497414587Z INFO: 'previous-logs' folder not found on ip-10-0-138-99.ec2.internal, skipping... 2026-05-29T03:22:18.561486377Z error: the server doesn't have a resource type "machineconfigs" 2026-05-29T03:22:18.578189475Z tar: Removing leading `/' from member names 2026-05-29T03:22:18.584048078Z INFO: HAProxy config collection complete. 2026-05-29T03:22:18.702254705Z Gathering data for ns/openshift-monitoring... 2026-05-29T03:22:18.727463666Z Wrote inspect data to must-gather. 2026-05-29T03:22:18.769243256Z INFO: 'previous-logs' folder not found on ip-10-0-140-28.ec2.internal, skipping... 2026-05-29T03:22:18.769465816Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-05-29T03:22:18.769524961Z INFO: Machine Config Daemon termination log collection complete. 2026-05-29T03:22:18.781489123Z INFO: Getting rules from prometheus-k8s-0 2026-05-29T03:22:18.786412251Z Wrote inspect data to must-gather. 2026-05-29T03:22:18.937818509Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-05-29T03:22:18.955664202Z Wrote inspect data to must-gather. 2026-05-29T03:22:18.966114728Z Wrote inspect data to must-gather. 2026-05-29T03:22:19.107703697Z error: the server doesn't have a resource type "kubeletconfigs" 2026-05-29T03:22:19.140633704Z INFO: Getting status/config from prometheus-k8s-0 2026-05-29T03:22:19.225116704Z INFO: INTERCONNECT MODE 2026-05-29T03:22:19.225150955Z INFO: Gathering ovn-kubernetes DBs 2026-05-29T03:22:19.297020613Z Wrote inspect data to must-gather. 2026-05-29T03:22:19.362730487Z INFO: Gathering OVN_Northbound from ovnkube-node-4shwz... 2026-05-29T03:22:19.363011929Z INFO: Gathering OVN_Northbound from ovnkube-node-psj9n... 2026-05-29T03:22:19.363257824Z INFO: Gathering OVN_Northbound from ovnkube-node-rw2sl... 2026-05-29T03:22:19.363545251Z INFO: Gathering OVN_Southbound from ovnkube-node-4shwz... 2026-05-29T03:22:19.363765287Z INFO: Gathering OVN_Southbound from ovnkube-node-psj9n... 2026-05-29T03:22:19.363993525Z INFO: Gathering OVN_Southbound from ovnkube-node-rw2sl... 2026-05-29T03:22:19.591774725Z Wrote inspect data to must-gather. 2026-05-29T03:22:19.647874165Z tar: Removing leading `/' from member names 2026-05-29T03:22:19.673798149Z INFO: Getting status/flags from prometheus-k8s-0 2026-05-29T03:22:19.686054958Z tar: Removing leading `/' from member names 2026-05-29T03:22:19.686136690Z tar: Removing leading `/' from member names 2026-05-29T03:22:19.696502323Z tar: Removing leading `/' from member names 2026-05-29T03:22:19.715620122Z tar: Removing leading `/' from member names 2026-05-29T03:22:19.720930886Z tar: Removing leading `/' from member names 2026-05-29T03:22:19.817472612Z Wrote inspect data to must-gather. 2026-05-29T03:22:19.904719209Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-05-29T03:22:20.013605759Z Wrote inspect data to must-gather. 2026-05-29T03:22:20.128545000Z INFO: Waiting for network log collection to complete ... 2026-05-29T03:22:20.128571194Z INFO: Waiting for ovnk database copies to complete ... 2026-05-29T03:22:20.128982791Z INFO: Copying ovnk databases complete. 2026-05-29T03:22:20.131156797Z 3.7M must-gather/network_logs/ovnk_database_store 2026-05-29T03:22:20.133697575Z ovnk_database_store/ 2026-05-29T03:22:20.133697575Z ovnk_database_store/ovnkube-node-rw2sl_sbdb 2026-05-29T03:22:20.135013062Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-05-29T03:22:20.187591742Z ovnk_database_store/ovnkube-node-psj9n_sbdb 2026-05-29T03:22:20.208664894Z ovnk_database_store/ovnkube-node-rw2sl_nbdb 2026-05-29T03:22:20.215530416Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-29T03:22:20.217039666Z ovnk_database_store/ovnkube-node-psj9n_nbdb 2026-05-29T03:22:20.224639676Z ovnk_database_store/ovnkube-node-4shwz_nbdb 2026-05-29T03:22:20.229322303Z Wrote inspect data to must-gather. 2026-05-29T03:22:20.233062789Z ovnk_database_store/ovnkube-node-4shwz_sbdb 2026-05-29T03:22:20.256931323Z INFO: Network log collection complete. 2026-05-29T03:22:20.368006380Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-05-29T03:22:20.417591877Z Wrote inspect data to must-gather. 2026-05-29T03:22:20.594994101Z Wrote inspect data to must-gather. 2026-05-29T03:22:20.736008070Z INFO: Getting status from alertmanager-main-0 2026-05-29T03:22:20.765116994Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-29T03:22:20.856287613Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-05-29T03:22:20.858370949Z INFO: Fallback to identify the container image from release info 2026-05-29T03:22:20.927731591Z Gathering data for ns/openshift-network-console... 2026-05-29T03:22:21.305740495Z Gathering data for ns/openshift-console-operator... 2026-05-29T03:22:21.362282234Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda 2026-05-29T03:22:21.481221138Z daemonset.apps/perf-node-gather-daemonset created 2026-05-29T03:22:21.601599950Z Waiting for performance profile collector pods to become ready: 1 2026-05-29T03:22:21.687991505Z Gathering data for ns/openshift-console... 2026-05-29T03:22:22.114048743Z Gathering data for ns/openshift-cluster-storage-operator... 2026-05-29T03:22:22.450948152Z Gathering data for ns/openshift-dns-operator... 2026-05-29T03:22:22.716988497Z Waiting for performance profile collector pods to become ready: 2 2026-05-29T03:22:22.718528804Z Gathering data for ns/openshift-dns... 2026-05-29T03:22:23.311632192Z Gathering data for ns/openshift-image-registry... 2026-05-29T03:22:23.797919822Z Gathering data for ns/openshift-ingress-operator... 2026-05-29T03:22:23.839016979Z Waiting for performance profile collector pods to become ready: 3 2026-05-29T03:22:23.942354982Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-05-29T03:22:24.110048768Z Gathering data for ns/openshift-ingress... 2026-05-29T03:22:24.408490599Z Gathering data for ns/openshift-ingress-canary... 2026-05-29T03:22:24.761534101Z Gathering data for ns/openshift-insights... 2026-05-29T03:22:24.937522118Z Waiting for performance profile collector pods to become ready: 4 2026-05-29T03:22:26.054707687Z Waiting for performance profile collector pods to become ready: 5 2026-05-29T03:22:26.699560692Z Gathering data for ns/openshift-config... 2026-05-29T03:22:27.004029166Z Gathering data for ns/openshift-config-managed... 2026-05-29T03:22:27.154231897Z Waiting for performance profile collector pods to become ready: 6 2026-05-29T03:22:27.432110044Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-05-29T03:22:27.737816831Z Gathering data for ns/openshift-kube-apiserver... 2026-05-29T03:22:28.055034769Z Gathering data for ns/openshift-kube-controller-manager... 2026-05-29T03:22:28.270378765Z Waiting for performance profile collector pods to become ready: 7 2026-05-29T03:22:28.325463720Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-05-29T03:22:28.664940213Z Gathering data for ns/openshift-kube-scheduler... 2026-05-29T03:22:28.946867786Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-05-29T03:22:28.952346235Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-05-29T03:22:29.219331946Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-05-29T03:22:29.388121857Z Waiting for performance profile collector pods to become ready: 8 2026-05-29T03:22:29.532438506Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-05-29T03:22:29.811727260Z Gathering data for ns/openshift-user-workload-monitoring... 2026-05-29T03:22:30.263361679Z Gathering data for ns/openshift-multus... 2026-05-29T03:22:30.493546538Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-05-29T03:22:30.713057436Z Collecting performance related data for node ip-10-0-138-99.ec2.internal 2026-05-29T03:22:30.719354328Z Collecting performance related data for node ip-10-0-140-28.ec2.internal 2026-05-29T03:22:30.726257554Z Collecting performance related data for node ip-10-0-137-122.ec2.internal 2026-05-29T03:22:31.794941506Z Gathering data for ns/openshift-ovn-kubernetes... 2026-05-29T03:22:33.490336138Z Gathering data for ns/openshift-host-network... 2026-05-29T03:22:33.804476683Z Gathering data for ns/openshift-network-diagnostics... 2026-05-29T03:22:33.955086057Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-05-29T03:22:34.292421880Z Gathering data for ns/openshift-network-node-identity... 2026-05-29T03:22:34.701531076Z Gathering data for ns/openshift-network-operator... 2026-05-29T03:22:35.143968050Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-05-29T03:22:35.408975283Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-05-29T03:22:35.863811064Z Gathering data for ns/openshift-apiserver-operator... 2026-05-29T03:22:36.144438236Z Gathering data for ns/openshift-apiserver... 2026-05-29T03:22:36.497212640Z Gathering data for ns/openshift-controller-manager-operator... 2026-05-29T03:22:36.740469612Z Gathering data for ns/openshift-controller-manager... 2026-05-29T03:22:36.991290011Z Gathering data for ns/openshift-cluster-samples-operator... 2026-05-29T03:22:37.648025507Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-05-29T03:22:37.926391961Z Gathering data for ns/openshift-service-ca-operator... 2026-05-29T03:22:38.264936712Z Gathering data for ns/openshift-service-ca... 2026-05-29T03:22:38.337027088Z Collecting kubelet logs for node ip-10-0-138-99.ec2.internal 2026-05-29T03:22:38.339325149Z Collecting kubelet logs for node ip-10-0-140-28.ec2.internal 2026-05-29T03:22:38.340933923Z Collecting kubelet logs for node ip-10-0-137-122.ec2.internal 2026-05-29T03:22:38.682574425Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-05-29T03:22:38.688110191Z INFO: Node performance data collection complete. 2026-05-29T03:22:38.709706849Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-05-29T03:22:38.960578591Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-05-29T03:22:39.689978445Z Wrote inspect data to must-gather. 2026-05-29T03:22:39.690260918Z error: inspection completed with the errors occurred while gathering data: 2026-05-29T03:22:39.690260918Z skipping gathering secrets/support due to error: secrets "support" not found 2026-05-29T03:22:39.817573448Z error: the server doesn't have a resource type "clusters" 2026-05-29T03:22:39.939806607Z Caches written to disk