2026-05-11T21:23:58.535013937Z [disk usage checker] Started
2026-05-11T21:23:58.538268684Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-11T21:23:58.763393701Z Gathering data for ns/openshift-cluster-version...
2026-05-11T21:23:58.930647353Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-11T21:23:59.024911951Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-11T21:23:59.126813885Z Gathering data for ns/default...
2026-05-11T21:23:59.462677140Z Gathering data for ns/openshift...
2026-05-11T21:23:59.923669453Z Gathering data for ns/kube-system...
2026-05-11T21:24:00.622166950Z Gathering data for ns/openshift-etcd...
2026-05-11T21:24:00.978561921Z Wrote inspect data to must-gather.
2026-05-11T21:24:00.978594848Z error: inspection completed with the errors occurred while gathering data:
2026-05-11T21:24:00.978594848Z     namespaces "assisted-installer" not found
2026-05-11T21:24:01.016158253Z Waiting on subprocesses to finish execution.
2026-05-11T21:24:01.021679128Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-11T21:24:01.026271987Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-11T21:24:01.029184456Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-11T21:24:01.047634777Z INFO: Collecting host service logs for crio
2026-05-11T21:24:01.047890549Z INFO: Collecting host service logs for kubelet
2026-05-11T21:24:01.048227338Z INFO: Collecting host service logs for rpm-ostreed
2026-05-11T21:24:01.048545221Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-11T21:24:01.048887495Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-11T21:24:01.049268801Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-11T21:24:01.049617799Z INFO: Collecting host service logs for NetworkManager
2026-05-11T21:24:01.049971631Z INFO: Collecting host service logs for openvswitch
2026-05-11T21:24:01.050305221Z INFO: Collecting host service logs for ovs-configuration
2026-05-11T21:24:01.050659408Z INFO: Collecting host service logs for ovsdb-server
2026-05-11T21:24:01.051019127Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-11T21:24:01.051516395Z INFO: Waiting for worker host service log collection to complete ...
2026-05-11T21:24:01.052493571Z INFO: Gathering HAProxy config files
2026-05-11T21:24:01.074533617Z INFO: Waiting for node performance related collection to complete ...
2026-05-11T21:24:01.549400723Z INFO: Collecting Insights Archives from insights-operator-544c98cc96-j75m5
2026-05-11T21:24:01.549474886Z insights-runtime-extractor-9cp9w
2026-05-11T21:24:01.549497453Z insights-runtime-extractor-jjq7d
2026-05-11T21:24:01.549518010Z insights-runtime-extractor-xkrjf
2026-05-11T21:24:01.827051668Z No resources found in openshift-etcd namespace.
2026-05-11T21:24:01.914916180Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-11T21:24:02.065078791Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-11T21:24:02.149085310Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-11T21:24:02.235267016Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-11T21:24:02.235362082Z INFO: on-disk MachineConfig config collection complete.
2026-05-11T21:24:02.322610303Z error: the server doesn't have a resource type "performanceprofile"
2026-05-11T21:24:02.344480273Z No resources found
2026-05-11T21:24:02.375405498Z INFO: "metallb-operator" not detected. Skipping.
2026-05-11T21:24:02.411510959Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-11T21:24:02.411510959Z See 'oc rsync -h' for help and examples
2026-05-11T21:24:02.413488249Z Wrote inspect data to must-gather.
2026-05-11T21:24:02.470675542Z INFO: Worker host service log collection to complete.
2026-05-11T21:24:02.572193871Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-11T21:24:02.638449334Z ERROR: No running kube-apiserver pods found
2026-05-11T21:24:02.965447916Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-11T21:24:02.982155873Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-11T21:24:03.058799399Z No resources found
2026-05-11T21:24:03.549219451Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-11T21:24:03.599741987Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-11T21:24:03.601397026Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-11T21:24:03.687624106Z INFO: 'previous-logs' folder not found on ip-10-0-128-58.ec2.internal, skipping...
2026-05-11T21:24:03.745511298Z Wrote inspect data to must-gather.
2026-05-11T21:24:03.773800932Z Wrote inspect data to must-gather.
2026-05-11T21:24:03.800494238Z Wrote inspect data to must-gather.
2026-05-11T21:24:03.825048377Z Wrote inspect data to must-gather.
2026-05-11T21:24:03.865294152Z Gathering data for ns/kuadrant-system...
2026-05-11T21:24:03.873550023Z Wrote inspect data to must-gather.
2026-05-11T21:24:03.913175694Z tar: Removing leading `/' from member names
2026-05-11T21:24:03.920322390Z INFO: HAProxy config collection complete.
2026-05-11T21:24:03.990877489Z INFO: Getting rules from prometheus-k8s-0
2026-05-11T21:24:04.091428134Z error: the server doesn't have a resource type "machineconfigs"
2026-05-11T21:24:04.135783411Z INFO: 'previous-logs' folder not found on ip-10-0-133-205.ec2.internal, skipping...
2026-05-11T21:24:04.209740348Z Wrote inspect data to must-gather.
2026-05-11T21:24:04.442085154Z Wrote inspect data to must-gather.
2026-05-11T21:24:04.469242257Z INFO: 'previous-logs' folder not found on ip-10-0-135-190.ec2.internal, skipping...
2026-05-11T21:24:04.470123571Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-11T21:24:04.470123571Z INFO: Machine Config Daemon termination log collection complete.
2026-05-11T21:24:04.535898097Z INFO: Getting status/config from prometheus-k8s-0
2026-05-11T21:24:04.606700356Z Wrote inspect data to must-gather.
2026-05-11T21:24:04.810430825Z Wrote inspect data to must-gather.
2026-05-11T21:24:05.070888179Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-11T21:24:05.084129870Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-11T21:24:05.113180245Z Wrote inspect data to must-gather.
2026-05-11T21:24:05.250047765Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-11T21:24:05.345282798Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-11T21:24:05.428658367Z Wrote inspect data to must-gather.
2026-05-11T21:24:05.444356736Z INFO: INTERCONNECT MODE
2026-05-11T21:24:05.444456850Z INFO: Gathering ovn-kubernetes DBs
2026-05-11T21:24:05.635470499Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-11T21:24:05.688171089Z INFO: Gathering OVN_Northbound from ovnkube-node-gkzk7...
2026-05-11T21:24:05.688446101Z INFO: Gathering OVN_Northbound from ovnkube-node-knpxn...
2026-05-11T21:24:05.688792107Z INFO: Gathering OVN_Northbound from ovnkube-node-svtmh...
2026-05-11T21:24:05.689141441Z INFO: Gathering OVN_Southbound from ovnkube-node-gkzk7...
2026-05-11T21:24:05.689433501Z INFO: Gathering OVN_Southbound from ovnkube-node-knpxn...
2026-05-11T21:24:05.689712725Z INFO: Gathering OVN_Southbound from ovnkube-node-svtmh...
2026-05-11T21:24:05.826277185Z W0511 21:24:05.826229    1184 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-11T21:24:05.862608934Z Wrote inspect data to must-gather.
2026-05-11T21:24:05.888350091Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-11T21:24:06.067567246Z tar: Removing leading `/' from member names
2026-05-11T21:24:06.096196993Z tar: Removing leading `/' from member names
2026-05-11T21:24:06.097503824Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-11T21:24:06.113135267Z tar: Removing leading `/' from member names
2026-05-11T21:24:06.162614791Z tar: Removing leading `/' from member names
2026-05-11T21:24:06.177473227Z tar: Removing leading `/' from member names
2026-05-11T21:24:06.231216891Z tar: Removing leading `/' from member names
2026-05-11T21:24:06.297071494Z Gathering data for ns/openshift-monitoring...
2026-05-11T21:24:06.399098961Z Wrote inspect data to must-gather.
2026-05-11T21:24:06.673694629Z INFO: Getting status from alertmanager-main-0
2026-05-11T21:24:06.783541546Z Wrote inspect data to must-gather.
2026-05-11T21:24:06.832775498Z INFO: Waiting for network log collection to complete ...
2026-05-11T21:24:06.832829602Z INFO: Waiting for ovnk database copies to complete ...
2026-05-11T21:24:06.833331078Z INFO: Copying ovnk databases complete.
2026-05-11T21:24:06.838722697Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-05-11T21:24:06.841659431Z ovnk_database_store/
2026-05-11T21:24:06.841659431Z ovnk_database_store/ovnkube-node-gkzk7_sbdb
2026-05-11T21:24:06.866818625Z ovnk_database_store/ovnkube-node-knpxn_nbdb
2026-05-11T21:24:06.880091428Z ovnk_database_store/ovnkube-node-gkzk7_nbdb
2026-05-11T21:24:06.909691896Z ovnk_database_store/ovnkube-node-knpxn_sbdb
2026-05-11T21:24:06.937562366Z ovnk_database_store/ovnkube-node-svtmh_sbdb
2026-05-11T21:24:06.965299353Z ovnk_database_store/ovnkube-node-svtmh_nbdb
2026-05-11T21:24:06.978210726Z INFO: Network log collection complete.
2026-05-11T21:24:07.081376964Z Wrote inspect data to must-gather.
2026-05-11T21:24:07.380490173Z Wrote inspect data to must-gather.
2026-05-11T21:24:07.687698832Z Wrote inspect data to must-gather.
2026-05-11T21:24:07.999946796Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-11T21:24:08.002261146Z INFO: Fallback to identify the container image from release info
2026-05-11T21:24:08.574662829Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e33dd3daea88bad2786194ceeeebe5712cd3befc2bbc8ace6a463156f9f7e36f
2026-05-11T21:24:08.589428678Z Gathering data for ns/openshift-network-console...
2026-05-11T21:24:08.620638751Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-11T21:24:08.726671874Z daemonset.apps/perf-node-gather-daemonset created
2026-05-11T21:24:08.867292287Z Waiting for performance profile collector pods to become ready: 1
2026-05-11T21:24:09.150146963Z Gathering data for ns/openshift-console-operator...
2026-05-11T21:24:09.627246756Z Gathering data for ns/openshift-console...
2026-05-11T21:24:10.016343645Z Waiting for performance profile collector pods to become ready: 2
2026-05-11T21:24:10.174678216Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-11T21:24:10.609759662Z Gathering data for ns/openshift-dns-operator...
2026-05-11T21:24:10.698877928Z Wrote inspect data to must-gather.
2026-05-11T21:24:10.933883288Z Gathering data for ns/openshift-dns...
2026-05-11T21:24:11.122534384Z Waiting for performance profile collector pods to become ready: 3
2026-05-11T21:24:11.543581782Z Gathering data for ns/openshift-image-registry...
2026-05-11T21:24:12.030254572Z Gathering data for ns/openshift-ingress-operator...
2026-05-11T21:24:12.239615868Z Waiting for performance profile collector pods to become ready: 4
2026-05-11T21:24:12.397748728Z Gathering data for ns/openshift-ingress...
2026-05-11T21:24:13.241799811Z Gathering data for ns/openshift-ingress-canary...
2026-05-11T21:24:13.358896208Z Waiting for performance profile collector pods to become ready: 5
2026-05-11T21:24:13.627447846Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-11T21:24:13.694211507Z Gathering data for ns/openshift-insights...
2026-05-11T21:24:14.468691407Z Waiting for performance profile collector pods to become ready: 6
2026-05-11T21:24:15.589857299Z Waiting for performance profile collector pods to become ready: 7
2026-05-11T21:24:15.662995664Z Gathering data for ns/opendatahub...
2026-05-11T21:24:16.703720871Z Waiting for performance profile collector pods to become ready: 8
2026-05-11T21:24:17.155550048Z Gathering data for ns/openshift-lws-operator...
2026-05-11T21:24:17.823827589Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-11T21:24:18.076321330Z Collecting performance related data for node ip-10-0-128-58.ec2.internal
2026-05-11T21:24:18.083215930Z Collecting performance related data for node ip-10-0-133-205.ec2.internal
2026-05-11T21:24:18.091442920Z Collecting performance related data for node ip-10-0-135-190.ec2.internal
2026-05-11T21:24:18.446725763Z Gathering data for ns/openshift-config...
2026-05-11T21:24:18.643610274Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-11T21:24:18.827568886Z Gathering data for ns/openshift-config-managed...
2026-05-11T21:24:19.430496993Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-11T21:24:19.785377350Z Gathering data for ns/openshift-kube-apiserver...
2026-05-11T21:24:20.073201535Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-11T21:24:20.411477158Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-11T21:24:20.781410242Z Gathering data for ns/openshift-kube-scheduler...
2026-05-11T21:24:21.121870687Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-11T21:24:21.474591246Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-11T21:24:21.816241193Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-11T21:24:22.146015913Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-11T21:24:22.707573567Z Gathering data for ns/openshift-multus...
2026-05-11T21:24:23.649713759Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-11T21:24:24.141595650Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-11T21:24:25.694641406Z Gathering data for ns/openshift-host-network...
2026-05-11T21:24:26.028397827Z Gathering data for ns/openshift-network-diagnostics...
2026-05-11T21:24:26.596524976Z Gathering data for ns/openshift-network-node-identity...
2026-05-11T21:24:26.738216987Z Collecting kubelet logs for node ip-10-0-128-58.ec2.internal
2026-05-11T21:24:26.740499222Z Collecting kubelet logs for node ip-10-0-133-205.ec2.internal
2026-05-11T21:24:26.742846932Z Collecting kubelet logs for node ip-10-0-135-190.ec2.internal
2026-05-11T21:24:27.147621101Z Gathering data for ns/openshift-network-operator...
2026-05-11T21:24:27.151465684Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-11T21:24:27.156494992Z INFO: Node performance data collection complete.
2026-05-11T21:24:27.549888919Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-11T21:24:27.831673617Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-11T21:24:28.298592693Z Gathering data for ns/openshift-apiserver-operator...
2026-05-11T21:24:28.626573291Z Gathering data for ns/openshift-apiserver...
2026-05-11T21:24:28.656250001Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-11T21:24:29.080538388Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-11T21:24:29.380932718Z Gathering data for ns/openshift-controller-manager...
2026-05-11T21:24:29.701933616Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-11T21:24:30.344254508Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-11T21:24:30.658654168Z Gathering data for ns/openshift-service-ca-operator...
2026-05-11T21:24:31.001172902Z Gathering data for ns/openshift-service-ca...
2026-05-11T21:24:31.400876081Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-11T21:24:32.375472609Z Wrote inspect data to must-gather.
2026-05-11T21:24:32.375514514Z error: inspection completed with the errors occurred while gathering data:
2026-05-11T21:24:32.375514514Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-11T21:24:32.507419495Z error: the server doesn't have a resource type "clusters"
2026-05-11T21:24:32.671447385Z Caches written to disk