2026-05-26T20:07:17.829925949Z [disk usage checker] Started
2026-05-26T20:07:17.833615100Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-26T20:07:18.057760113Z Gathering data for ns/openshift-cluster-version...
2026-05-26T20:07:18.215530503Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T20:07:18.307556229Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T20:07:18.409281719Z Gathering data for ns/default...
2026-05-26T20:07:18.754171372Z Gathering data for ns/openshift...
2026-05-26T20:07:19.234123330Z Gathering data for ns/kube-system...
2026-05-26T20:07:19.961441344Z Gathering data for ns/openshift-etcd...
2026-05-26T20:07:20.314835742Z Wrote inspect data to must-gather.
2026-05-26T20:07:20.314870017Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T20:07:20.314870017Z     namespaces "assisted-installer" not found
2026-05-26T20:07:20.362219777Z Waiting on subprocesses to finish execution.
2026-05-26T20:07:20.368788853Z INFO: Collecting host service logs for crio
2026-05-26T20:07:20.368832218Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-26T20:07:20.369074360Z INFO: Collecting host service logs for kubelet
2026-05-26T20:07:20.369477026Z INFO: Collecting host service logs for rpm-ostreed
2026-05-26T20:07:20.369770407Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-26T20:07:20.370081874Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-26T20:07:20.370404045Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-26T20:07:20.370702908Z INFO: Collecting host service logs for NetworkManager
2026-05-26T20:07:20.371010459Z INFO: Collecting host service logs for openvswitch
2026-05-26T20:07:20.371505811Z INFO: Collecting host service logs for ovs-configuration
2026-05-26T20:07:20.371834420Z INFO: Collecting host service logs for ovsdb-server
2026-05-26T20:07:20.372132251Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-26T20:07:20.372505129Z INFO: Waiting for worker host service log collection to complete ...
2026-05-26T20:07:20.394875752Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-26T20:07:20.430817822Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-26T20:07:20.480008424Z INFO: Gathering HAProxy config files
2026-05-26T20:07:20.572954370Z INFO: Waiting for node performance related collection to complete ...
2026-05-26T20:07:20.725164606Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-7kwtj
2026-05-26T20:07:20.725219072Z insights-runtime-extractor-5lt74
2026-05-26T20:07:20.725239272Z insights-runtime-extractor-kcfsc
2026-05-26T20:07:20.725258551Z insights-runtime-extractor-x978b
2026-05-26T20:07:21.244103196Z INFO: "metallb-operator" not detected. Skipping.
2026-05-26T20:07:21.325183933Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-26T20:07:21.325183933Z See 'oc rsync -h' for help and examples
2026-05-26T20:07:21.412233932Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-26T20:07:21.412445502Z INFO: on-disk MachineConfig config collection complete.
2026-05-26T20:07:21.577346907Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-26T20:07:21.590713697Z No resources found in openshift-etcd namespace.
2026-05-26T20:07:21.621554398Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-26T20:07:21.672730171Z error: the server doesn't have a resource type "performanceprofile"
2026-05-26T20:07:21.676779883Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-26T20:07:21.848613037Z Wrote inspect data to must-gather.
2026-05-26T20:07:21.851889691Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-26T20:07:21.880133241Z INFO: Worker host service log collection to complete.
2026-05-26T20:07:21.949139497Z No resources found
2026-05-26T20:07:22.266003384Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-26T20:07:22.276726792Z ERROR: No running kube-apiserver pods found
2026-05-26T20:07:22.542435888Z INFO: 'previous-logs' folder not found on ip-10-0-131-212.ec2.internal, skipping...
2026-05-26T20:07:22.723489923Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T20:07:22.756277268Z Wrote inspect data to must-gather.
2026-05-26T20:07:22.809520189Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-26T20:07:22.956547548Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-26T20:07:23.002569620Z Wrote inspect data to must-gather.
2026-05-26T20:07:23.020731905Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-26T20:07:23.055182480Z No resources found
2026-05-26T20:07:23.141266466Z tar: Removing leading `/' from member names
2026-05-26T20:07:23.150310656Z INFO: HAProxy config collection complete.
2026-05-26T20:07:23.194743214Z Wrote inspect data to must-gather.
2026-05-26T20:07:23.314212802Z INFO: 'previous-logs' folder not found on ip-10-0-139-192.ec2.internal, skipping...
2026-05-26T20:07:23.320842642Z Wrote inspect data to must-gather.
2026-05-26T20:07:23.390791277Z error: the server doesn't have a resource type "machineconfigs"
2026-05-26T20:07:23.408514603Z Wrote inspect data to must-gather.
2026-05-26T20:07:23.419875261Z Gathering data for ns/kuadrant-system...
2026-05-26T20:07:23.644602038Z INFO: Getting rules from prometheus-k8s-0
2026-05-26T20:07:23.693597444Z INFO: 'previous-logs' folder not found on ip-10-0-143-77.ec2.internal, skipping...
2026-05-26T20:07:23.693708819Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-26T20:07:23.693757865Z INFO: Machine Config Daemon termination log collection complete.
2026-05-26T20:07:23.731588446Z Wrote inspect data to must-gather.
2026-05-26T20:07:23.796921476Z Wrote inspect data to must-gather.
2026-05-26T20:07:24.092636843Z Wrote inspect data to must-gather.
2026-05-26T20:07:24.171603902Z INFO: Getting status/config from prometheus-k8s-0
2026-05-26T20:07:24.172411351Z Wrote inspect data to must-gather.
2026-05-26T20:07:24.319416658Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-26T20:07:24.552263273Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T20:07:24.552881667Z Wrote inspect data to must-gather.
2026-05-26T20:07:24.641607945Z Wrote inspect data to must-gather.
2026-05-26T20:07:24.648891096Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-26T20:07:24.964707283Z INFO: INTERCONNECT MODE
2026-05-26T20:07:24.964801741Z INFO: Gathering ovn-kubernetes DBs
2026-05-26T20:07:24.969176111Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-26T20:07:24.969429639Z Wrote inspect data to must-gather.
2026-05-26T20:07:25.170112968Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T20:07:25.202964765Z INFO: Gathering OVN_Northbound from ovnkube-node-59ptn...
2026-05-26T20:07:25.203245596Z INFO: Gathering OVN_Northbound from ovnkube-node-k89dv...
2026-05-26T20:07:25.203529810Z INFO: Gathering OVN_Northbound from ovnkube-node-shbc8...
2026-05-26T20:07:25.203818348Z INFO: Gathering OVN_Southbound from ovnkube-node-59ptn...
2026-05-26T20:07:25.204065771Z INFO: Gathering OVN_Southbound from ovnkube-node-k89dv...
2026-05-26T20:07:25.204316528Z INFO: Gathering OVN_Southbound from ovnkube-node-shbc8...
2026-05-26T20:07:25.257444222Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-26T20:07:25.484771219Z Gathering data for ns/openshift-monitoring...
2026-05-26T20:07:25.504762671Z Wrote inspect data to must-gather.
2026-05-26T20:07:25.560510175Z tar: Removing leading `/' from member names
2026-05-26T20:07:25.585630836Z tar: Removing leading `/' from member names
2026-05-26T20:07:25.633416967Z tar: Removing leading `/' from member names
2026-05-26T20:07:25.637737151Z tar: Removing leading `/' from member names
2026-05-26T20:07:25.652716383Z tar: Removing leading `/' from member names
2026-05-26T20:07:25.766343484Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-26T20:07:25.933509433Z tar: Removing leading `/' from member names
2026-05-26T20:07:25.954291211Z Wrote inspect data to must-gather.
2026-05-26T20:07:26.224442901Z INFO: Getting status from alertmanager-main-0
2026-05-26T20:07:26.353138665Z INFO: Waiting for network log collection to complete ...
2026-05-26T20:07:26.353191158Z INFO: Waiting for ovnk database copies to complete ...
2026-05-26T20:07:26.353689602Z INFO: Copying ovnk databases complete.
2026-05-26T20:07:26.355350531Z 4.4M	must-gather/network_logs/ovnk_database_store
2026-05-26T20:07:26.357645313Z ovnk_database_store/
2026-05-26T20:07:26.357645313Z ovnk_database_store/ovnkube-node-k89dv_nbdb
2026-05-26T20:07:26.384602857Z Wrote inspect data to must-gather.
2026-05-26T20:07:26.404191939Z ovnk_database_store/ovnkube-node-shbc8_nbdb
2026-05-26T20:07:26.414994625Z ovnk_database_store/ovnkube-node-59ptn_nbdb
2026-05-26T20:07:26.427927536Z ovnk_database_store/ovnkube-node-59ptn_sbdb
2026-05-26T20:07:26.452790157Z ovnk_database_store/ovnkube-node-shbc8_sbdb
2026-05-26T20:07:26.473475591Z ovnk_database_store/ovnkube-node-k89dv_sbdb
2026-05-26T20:07:26.507495441Z INFO: Network log collection complete.
2026-05-26T20:07:26.728777621Z Wrote inspect data to must-gather.
2026-05-26T20:07:27.022922004Z Wrote inspect data to must-gather.
2026-05-26T20:07:27.316852250Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-26T20:07:27.319128746Z INFO: Fallback to identify the container image from release info
2026-05-26T20:07:27.716948413Z Gathering data for ns/openshift-network-console...
2026-05-26T20:07:28.019858837Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-26T20:07:28.060014759Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-26T20:07:28.167467080Z daemonset.apps/perf-node-gather-daemonset created
2026-05-26T20:07:28.236593337Z Gathering data for ns/openshift-console-operator...
2026-05-26T20:07:28.328745281Z Waiting for performance profile collector pods to become ready: 1
2026-05-26T20:07:28.733082656Z Gathering data for ns/openshift-console...
2026-05-26T20:07:29.230114113Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-26T20:07:29.480205408Z Waiting for performance profile collector pods to become ready: 2
2026-05-26T20:07:29.686439944Z Gathering data for ns/openshift-dns-operator...
2026-05-26T20:07:29.968270176Z Wrote inspect data to must-gather.
2026-05-26T20:07:30.023674196Z Gathering data for ns/openshift-dns...
2026-05-26T20:07:30.620003087Z Waiting for performance profile collector pods to become ready: 3
2026-05-26T20:07:30.620742436Z Gathering data for ns/openshift-image-registry...
2026-05-26T20:07:31.097718886Z Gathering data for ns/openshift-ingress-operator...
2026-05-26T20:07:31.457529502Z Gathering data for ns/openshift-ingress...
2026-05-26T20:07:31.740664486Z Waiting for performance profile collector pods to become ready: 4
2026-05-26T20:07:32.242847132Z Gathering data for ns/openshift-ingress-canary...
2026-05-26T20:07:32.679712699Z Gathering data for ns/openshift-insights...
2026-05-26T20:07:32.849849826Z Waiting for performance profile collector pods to become ready: 5
2026-05-26T20:07:33.064627305Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-26T20:07:33.964466004Z Waiting for performance profile collector pods to become ready: 6
2026-05-26T20:07:34.691422990Z Gathering data for ns/opendatahub...
2026-05-26T20:07:35.107906100Z Waiting for performance profile collector pods to become ready: 7
2026-05-26T20:07:36.241901319Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-26T20:07:36.280004259Z Gathering data for ns/openshift-lws-operator...
2026-05-26T20:07:36.516674799Z Collecting performance related data for node ip-10-0-143-77.ec2.internal
2026-05-26T20:07:36.525947918Z Collecting performance related data for node ip-10-0-131-212.ec2.internal
2026-05-26T20:07:36.533769880Z Collecting performance related data for node ip-10-0-139-192.ec2.internal
2026-05-26T20:07:37.780358100Z Gathering data for ns/openshift-config...
2026-05-26T20:07:38.072118136Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-26T20:07:38.128024738Z Gathering data for ns/openshift-config-managed...
2026-05-26T20:07:38.567021254Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-26T20:07:38.885309896Z Gathering data for ns/openshift-kube-apiserver...
2026-05-26T20:07:39.234501167Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-26T20:07:39.721888074Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-26T20:07:40.075958697Z Gathering data for ns/openshift-kube-scheduler...
2026-05-26T20:07:40.380901880Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-26T20:07:40.721260575Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-26T20:07:41.118999381Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-26T20:07:41.455304847Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-26T20:07:42.021254873Z Gathering data for ns/openshift-multus...
2026-05-26T20:07:43.078922646Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-26T20:07:43.409688621Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-26T20:07:44.857039130Z Gathering data for ns/openshift-host-network...
2026-05-26T20:07:45.221677470Z Gathering data for ns/openshift-network-diagnostics...
2026-05-26T20:07:45.443181187Z Collecting kubelet logs for node ip-10-0-143-77.ec2.internal
2026-05-26T20:07:45.445025218Z Collecting kubelet logs for node ip-10-0-131-212.ec2.internal
2026-05-26T20:07:45.446796804Z Collecting kubelet logs for node ip-10-0-139-192.ec2.internal
2026-05-26T20:07:45.864970907Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-26T20:07:45.871866138Z INFO: Node performance data collection complete.
2026-05-26T20:07:45.885343783Z Gathering data for ns/openshift-network-node-identity...
2026-05-26T20:07:46.368697824Z Gathering data for ns/openshift-network-operator...
2026-05-26T20:07:46.771439902Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-26T20:07:47.068664831Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-26T20:07:47.560720038Z Gathering data for ns/openshift-apiserver-operator...
2026-05-26T20:07:47.903069401Z Gathering data for ns/openshift-apiserver...
2026-05-26T20:07:48.086423885Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-26T20:07:48.340476903Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-26T20:07:48.635665563Z Gathering data for ns/openshift-controller-manager...
2026-05-26T20:07:48.938231729Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-26T20:07:49.614790436Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-26T20:07:49.946253600Z Gathering data for ns/openshift-service-ca-operator...
2026-05-26T20:07:50.283365375Z Gathering data for ns/openshift-service-ca...
2026-05-26T20:07:50.667159612Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-26T20:07:51.664598430Z Wrote inspect data to must-gather.
2026-05-26T20:07:51.664644470Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T20:07:51.664644470Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-26T20:07:51.802687650Z error: the server doesn't have a resource type "clusters"
2026-05-26T20:07:51.986550271Z Caches written to disk