2026-04-22T18:24:48.392195200Z [disk usage checker] Started
2026-04-22T18:24:48.395545291Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-22T18:24:48.623860615Z Gathering data for ns/openshift-cluster-version...
2026-04-22T18:24:48.775446506Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T18:24:48.866785903Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T18:24:48.961490841Z Gathering data for ns/default...
2026-04-22T18:24:49.321407253Z Gathering data for ns/openshift...
2026-04-22T18:24:49.778735735Z Gathering data for ns/kube-system...
2026-04-22T18:24:50.460876155Z Gathering data for ns/openshift-etcd...
2026-04-22T18:24:50.812803381Z Wrote inspect data to must-gather.
2026-04-22T18:24:50.812840550Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T18:24:50.812840550Z     namespaces "assisted-installer" not found
2026-04-22T18:24:51.014770505Z Waiting on subprocesses to finish execution.
2026-04-22T18:24:51.028092967Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T18:24:51.028405538Z INFO: Gathering HAProxy config files
2026-04-22T18:24:51.031886070Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T18:24:51.031980143Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T18:24:51.081856287Z INFO: Collecting host service logs for crio
2026-04-22T18:24:51.082231016Z INFO: Collecting host service logs for kubelet
2026-04-22T18:24:51.082620482Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T18:24:51.083040141Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T18:24:51.083410136Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T18:24:51.083792972Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T18:24:51.086488896Z INFO: Collecting host service logs for NetworkManager
2026-04-22T18:24:51.086860127Z INFO: Collecting host service logs for openvswitch
2026-04-22T18:24:51.087184805Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T18:24:51.087516333Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T18:24:51.087882149Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T18:24:51.088305726Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T18:24:51.173688707Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T18:24:51.589934767Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T18:24:51.849790289Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-vkb44
2026-04-22T18:24:51.849847267Z insights-runtime-extractor-5skr5
2026-04-22T18:24:51.849871151Z insights-runtime-extractor-77tjc
2026-04-22T18:24:51.849893891Z insights-runtime-extractor-clh2q
2026-04-22T18:24:52.089201293Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T18:24:52.090876890Z No resources found in openshift-etcd namespace.
2026-04-22T18:24:52.200112000Z Wrote inspect data to must-gather.
2026-04-22T18:24:52.220071021Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T18:24:52.220100648Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T18:24:52.389308049Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T18:24:52.407376992Z No resources found
2026-04-22T18:24:52.910526649Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T18:24:52.928444018Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T18:24:53.044885208Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T18:24:53.108014694Z INFO: Worker host service log collection to complete.
2026-04-22T18:24:53.160088865Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T18:24:53.160088865Z See 'oc rsync -h' for help and examples
2026-04-22T18:24:53.205838827Z ERROR: No running kube-apiserver pods found
2026-04-22T18:24:53.242886692Z No resources found
2026-04-22T18:24:53.243885300Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T18:24:53.259988013Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T18:24:53.324065741Z Wrote inspect data to must-gather.
2026-04-22T18:24:53.337018746Z Wrote inspect data to must-gather.
2026-04-22T18:24:53.349288554Z Wrote inspect data to must-gather.
2026-04-22T18:24:53.482704698Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-22T18:24:53.583671382Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T18:24:53.590730266Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T18:24:53.684828501Z Wrote inspect data to must-gather.
2026-04-22T18:24:53.691979085Z INFO: 'previous-logs' folder not found on ip-10-0-128-219.ec2.internal, skipping...
2026-04-22T18:24:53.833477399Z Wrote inspect data to must-gather.
2026-04-22T18:24:53.864754394Z tar: Removing leading `/' from member names
2026-04-22T18:24:53.872261625Z INFO: HAProxy config collection complete.
2026-04-22T18:24:53.937763476Z Gathering data for ns/kuadrant-system...
2026-04-22T18:24:54.039042516Z Wrote inspect data to must-gather.
2026-04-22T18:24:54.083022719Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T18:24:54.101144901Z INFO: 'previous-logs' folder not found on ip-10-0-131-69.ec2.internal, skipping...
2026-04-22T18:24:54.210607556Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T18:24:54.434762052Z Wrote inspect data to must-gather.
2026-04-22T18:24:54.466282229Z Wrote inspect data to must-gather.
2026-04-22T18:24:54.540276445Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T18:24:54.555641288Z INFO: 'previous-logs' folder not found on ip-10-0-142-118.ec2.internal, skipping...
2026-04-22T18:24:54.555754855Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T18:24:54.555808410Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T18:24:54.770008704Z Wrote inspect data to must-gather.
2026-04-22T18:24:54.803426320Z W0422 18:24:54.803372    1067 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-22T18:24:54.810586956Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T18:24:54.828849152Z Wrote inspect data to must-gather.
2026-04-22T18:24:54.865114404Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T18:24:55.007469304Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T18:24:55.121997342Z Gathering data for ns/openshift-monitoring...
2026-04-22T18:24:55.263820292Z INFO: INTERCONNECT MODE
2026-04-22T18:24:55.263914797Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T18:24:55.359890682Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T18:24:55.363693260Z Wrote inspect data to must-gather.
2026-04-22T18:24:55.495050481Z INFO: Gathering OVN_Northbound from ovnkube-node-9vgh6...
2026-04-22T18:24:55.495314789Z INFO: Gathering OVN_Northbound from ovnkube-node-fvkcv...
2026-04-22T18:24:55.495615735Z INFO: Gathering OVN_Northbound from ovnkube-node-t6kbb...
2026-04-22T18:24:55.495971913Z INFO: Gathering OVN_Southbound from ovnkube-node-9vgh6...
2026-04-22T18:24:55.496256688Z INFO: Gathering OVN_Southbound from ovnkube-node-fvkcv...
2026-04-22T18:24:55.496537837Z INFO: Gathering OVN_Southbound from ovnkube-node-t6kbb...
2026-04-22T18:24:55.741665424Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T18:24:55.837487907Z Wrote inspect data to must-gather.
2026-04-22T18:24:55.839649808Z tar: Removing leading `/' from member names
2026-04-22T18:24:55.842559293Z tar: Removing leading `/' from member names
2026-04-22T18:24:55.878550603Z tar: Removing leading `/' from member names
2026-04-22T18:24:55.910453927Z tar: Removing leading `/' from member names
2026-04-22T18:24:55.940489945Z tar: Removing leading `/' from member names
2026-04-22T18:24:55.942870704Z tar: Removing leading `/' from member names
2026-04-22T18:24:56.137940211Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T18:24:56.205679172Z Wrote inspect data to must-gather.
2026-04-22T18:24:56.505072445Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T18:24:56.611385950Z Wrote inspect data to must-gather.
2026-04-22T18:24:56.644312827Z INFO: Waiting for network log collection to complete ...
2026-04-22T18:24:56.644481126Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T18:24:56.644912181Z INFO: Copying ovnk databases complete.
2026-04-22T18:24:56.646586920Z 4.9M	must-gather/network_logs/ovnk_database_store
2026-04-22T18:24:56.648922794Z ovnk_database_store/
2026-04-22T18:24:56.648975126Z ovnk_database_store/ovnkube-node-fvkcv_nbdb
2026-04-22T18:24:56.661202053Z ovnk_database_store/ovnkube-node-9vgh6_sbdb
2026-04-22T18:24:56.696183153Z ovnk_database_store/ovnkube-node-9vgh6_nbdb
2026-04-22T18:24:56.707448872Z ovnk_database_store/ovnkube-node-t6kbb_nbdb
2026-04-22T18:24:56.717777381Z ovnk_database_store/ovnkube-node-fvkcv_sbdb
2026-04-22T18:24:56.746963435Z ovnk_database_store/ovnkube-node-t6kbb_sbdb
2026-04-22T18:24:56.778021193Z INFO: Network log collection complete.
2026-04-22T18:24:56.933208323Z Wrote inspect data to must-gather.
2026-04-22T18:24:56.969352124Z INFO: Getting status from alertmanager-main-0
2026-04-22T18:24:57.220310529Z W0422 18:24:57.220247    1399 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-22T18:24:57.229031737Z Wrote inspect data to must-gather.
2026-04-22T18:24:57.494995237Z Gathering data for ns/openshift-network-console...
2026-04-22T18:24:57.581237290Z Wrote inspect data to must-gather.
2026-04-22T18:24:57.889138039Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T18:24:57.893270776Z INFO: Fallback to identify the container image from release info
2026-04-22T18:24:57.995379853Z Gathering data for ns/openshift-console-operator...
2026-04-22T18:24:58.428102454Z Gathering data for ns/openshift-console...
2026-04-22T18:24:58.518637864Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T18:24:58.539102471Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-22T18:24:58.680612802Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T18:24:58.819052850Z Wrote inspect data to must-gather.
2026-04-22T18:24:58.843064957Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T18:24:58.950874830Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T18:24:59.332954922Z Gathering data for ns/openshift-dns-operator...
2026-04-22T18:24:59.658203062Z Gathering data for ns/openshift-dns...
2026-04-22T18:24:59.986685249Z Waiting for performance profile collector pods to become ready: 2
2026-04-22T18:25:00.231014747Z Gathering data for ns/openshift-image-registry...
2026-04-22T18:25:00.720646284Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T18:25:01.068714183Z Gathering data for ns/openshift-ingress...
2026-04-22T18:25:01.126451658Z Waiting for performance profile collector pods to become ready: 3
2026-04-22T18:25:01.605109503Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T18:25:02.015765838Z Gathering data for ns/openshift-insights...
2026-04-22T18:25:02.249956478Z Waiting for performance profile collector pods to become ready: 4
2026-04-22T18:25:03.374989586Z Waiting for performance profile collector pods to become ready: 5
2026-04-22T18:25:03.543974126Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-22T18:25:04.505526663Z Waiting for performance profile collector pods to become ready: 6
2026-04-22T18:25:04.668343855Z Gathering data for ns/openshift-lws-operator...
2026-04-22T18:25:05.307713886Z Gathering data for ns/kserve...
2026-04-22T18:25:05.646658194Z Waiting for performance profile collector pods to become ready: 7
2026-04-22T18:25:06.783167875Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T18:25:07.040669077Z Collecting performance related data for node ip-10-0-128-219.ec2.internal
2026-04-22T18:25:07.047201360Z Collecting performance related data for node ip-10-0-142-118.ec2.internal
2026-04-22T18:25:07.054693370Z Collecting performance related data for node ip-10-0-131-69.ec2.internal
2026-04-22T18:25:07.094885557Z Gathering data for ns/openshift-config...
2026-04-22T18:25:07.480086556Z Gathering data for ns/openshift-config-managed...
2026-04-22T18:25:08.032969803Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T18:25:08.422048267Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T18:25:08.549257628Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-22T18:25:08.780544558Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T18:25:09.086489700Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T18:25:09.396354865Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T18:25:09.739831959Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T18:25:10.128558595Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T18:25:10.453377104Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T18:25:10.779967011Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T18:25:11.332604723Z Gathering data for ns/openshift-multus...
2026-04-22T18:25:12.987305028Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T18:25:13.554704120Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-22T18:25:14.547640140Z Gathering data for ns/openshift-host-network...
2026-04-22T18:25:14.958218939Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T18:25:15.496067703Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T18:25:15.945174018Z Gathering data for ns/openshift-network-operator...
2026-04-22T18:25:15.949159465Z Collecting kubelet logs for node ip-10-0-128-219.ec2.internal
2026-04-22T18:25:15.950940931Z Collecting kubelet logs for node ip-10-0-142-118.ec2.internal
2026-04-22T18:25:15.953047255Z Collecting kubelet logs for node ip-10-0-131-69.ec2.internal
2026-04-22T18:25:16.356855050Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T18:25:16.361925059Z INFO: Node performance data collection complete.
2026-04-22T18:25:16.411929279Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T18:25:16.723227565Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T18:25:17.164469949Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T18:25:17.500109732Z Gathering data for ns/openshift-apiserver...
2026-04-22T18:25:17.948656229Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T18:25:18.240862131Z Gathering data for ns/openshift-controller-manager...
2026-04-22T18:25:18.519313652Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T18:25:18.559349971Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-22T18:25:19.178977716Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T18:25:19.468076719Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T18:25:19.783095000Z Gathering data for ns/openshift-service-ca...
2026-04-22T18:25:20.140999953Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T18:25:21.036931851Z Wrote inspect data to must-gather.
2026-04-22T18:25:21.036977964Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T18:25:21.036977964Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T18:25:21.172308225Z error: the server doesn't have a resource type "clusters"
2026-04-22T18:25:21.348544933Z Caches written to disk