2026-04-24T22:49:17.183001744Z [disk usage checker] Started
2026-04-24T22:49:17.186283221Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T22:49:17.402708613Z Gathering data for ns/openshift-cluster-version...
2026-04-24T22:49:17.534366260Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:49:17.621048012Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:49:17.705069284Z Gathering data for ns/default...
2026-04-24T22:49:18.008598764Z Gathering data for ns/openshift...
2026-04-24T22:49:18.430495648Z Gathering data for ns/kube-system...
2026-04-24T22:49:19.235659476Z Gathering data for ns/openshift-etcd...
2026-04-24T22:49:19.555314064Z Wrote inspect data to must-gather.
2026-04-24T22:49:19.555407627Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:49:19.555407627Z     namespaces "assisted-installer" not found
2026-04-24T22:49:19.662621857Z Waiting on subprocesses to finish execution.
2026-04-24T22:49:19.670175214Z INFO: Gathering HAProxy config files
2026-04-24T22:49:19.681243677Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T22:49:19.687174271Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T22:49:19.688114577Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T22:49:19.690162105Z INFO: Collecting host service logs for crio
2026-04-24T22:49:19.690496907Z INFO: Collecting host service logs for kubelet
2026-04-24T22:49:19.690824738Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T22:49:19.690844405Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T22:49:19.691175565Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T22:49:19.691478798Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T22:49:19.691820356Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T22:49:19.692168308Z INFO: Collecting host service logs for NetworkManager
2026-04-24T22:49:19.692520211Z INFO: Collecting host service logs for openvswitch
2026-04-24T22:49:19.692892151Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T22:49:19.693273389Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T22:49:19.693625606Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T22:49:19.694057654Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T22:49:20.055003177Z No resources found
2026-04-24T22:49:20.149949631Z No resources found in openshift-etcd namespace.
2026-04-24T22:49:20.210681861Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T22:49:20.258825300Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T22:49:20.514548069Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T22:49:20.529119627Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-lj9nb
2026-04-24T22:49:20.529314142Z insights-runtime-extractor-dldk9
2026-04-24T22:49:20.529339902Z insights-runtime-extractor-mdrlq
2026-04-24T22:49:20.529363597Z insights-runtime-extractor-xs49w
2026-04-24T22:49:20.779211265Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T22:49:21.052520795Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T22:49:21.052520795Z See 'oc rsync -h' for help and examples
2026-04-24T22:49:21.314629965Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T22:49:21.314660840Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T22:49:21.335844933Z Wrote inspect data to must-gather.
2026-04-24T22:49:21.472749540Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T22:49:21.548564387Z ERROR: No running kube-apiserver pods found
2026-04-24T22:49:21.576745764Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T22:49:21.628680024Z Wrote inspect data to must-gather.
2026-04-24T22:49:21.717766360Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:49:21.833998130Z Wrote inspect data to must-gather.
2026-04-24T22:49:21.872472230Z No resources found
2026-04-24T22:49:21.877389524Z INFO: Worker host service log collection to complete.
2026-04-24T22:49:21.887277218Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T22:49:22.037897278Z Wrote inspect data to must-gather.
2026-04-24T22:49:22.216497317Z Wrote inspect data to must-gather.
2026-04-24T22:49:22.230757855Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T22:49:22.367957064Z Wrote inspect data to must-gather.
2026-04-24T22:49:22.397721917Z Wrote inspect data to must-gather.
2026-04-24T22:49:22.439918128Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T22:49:22.474391206Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T22:49:22.651067374Z INFO: 'previous-logs' folder not found on ip-10-0-129-176.ec2.internal, skipping...
2026-04-24T22:49:22.695425286Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T22:49:22.719980549Z Gathering data for ns/kuadrant-system...
2026-04-24T22:49:22.746681134Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T22:49:22.768676324Z tar: Removing leading `/' from member names
2026-04-24T22:49:22.769935555Z Wrote inspect data to must-gather.
2026-04-24T22:49:22.773607165Z INFO: HAProxy config collection complete.
2026-04-24T22:49:23.070970176Z Wrote inspect data to must-gather.
2026-04-24T22:49:23.078096010Z INFO: 'previous-logs' folder not found on ip-10-0-133-9.ec2.internal, skipping...
2026-04-24T22:49:23.152121974Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:49:23.191797045Z Wrote inspect data to must-gather.
2026-04-24T22:49:23.226352114Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T22:49:23.420064504Z Wrote inspect data to must-gather.
2026-04-24T22:49:23.459032867Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:49:23.495045242Z INFO: 'previous-logs' folder not found on ip-10-0-134-101.ec2.internal, skipping...
2026-04-24T22:49:23.495150260Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T22:49:23.495201350Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T22:49:23.656480232Z Gathering data for ns/openshift-monitoring...
2026-04-24T22:49:23.660159965Z INFO: INTERCONNECT MODE
2026-04-24T22:49:23.660185368Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T22:49:23.666868941Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T22:49:23.744889764Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T22:49:23.873290943Z INFO: Gathering OVN_Northbound from ovnkube-node-46t57...
2026-04-24T22:49:23.873539088Z INFO: Gathering OVN_Northbound from ovnkube-node-bbfmd...
2026-04-24T22:49:23.873932461Z INFO: Gathering OVN_Northbound from ovnkube-node-xsvks...
2026-04-24T22:49:23.874219570Z INFO: Gathering OVN_Southbound from ovnkube-node-46t57...
2026-04-24T22:49:23.874509523Z INFO: Gathering OVN_Southbound from ovnkube-node-bbfmd...
2026-04-24T22:49:23.874851189Z INFO: Gathering OVN_Southbound from ovnkube-node-xsvks...
2026-04-24T22:49:24.040102597Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T22:49:24.070100059Z Wrote inspect data to must-gather.
2026-04-24T22:49:24.194926411Z tar: Removing leading `/' from member names
2026-04-24T22:49:24.204587931Z tar: Removing leading `/' from member names
2026-04-24T22:49:24.206413811Z tar: Removing leading `/' from member names
2026-04-24T22:49:24.240360378Z tar: Removing leading `/' from member names
2026-04-24T22:49:24.296114584Z tar: Removing leading `/' from member names
2026-04-24T22:49:24.313414422Z tar: Removing leading `/' from member names
2026-04-24T22:49:24.398858217Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T22:49:24.456179611Z Wrote inspect data to must-gather.
2026-04-24T22:49:24.663717732Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T22:49:24.762117255Z Wrote inspect data to must-gather.
2026-04-24T22:49:24.895653450Z INFO: Waiting for network log collection to complete ...
2026-04-24T22:49:24.895766306Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T22:49:24.896331667Z INFO: Copying ovnk databases complete.
2026-04-24T22:49:24.898685336Z 3.2M	must-gather/network_logs/ovnk_database_store
2026-04-24T22:49:24.903198006Z ovnk_database_store/
2026-04-24T22:49:24.903251615Z ovnk_database_store/ovnkube-node-bbfmd_sbdb
2026-04-24T22:49:24.926757626Z ovnk_database_store/ovnkube-node-46t57_sbdb
2026-04-24T22:49:24.960602774Z ovnk_database_store/ovnkube-node-xsvks_sbdb
2026-04-24T22:49:24.983189639Z ovnk_database_store/ovnkube-node-xsvks_nbdb
2026-04-24T22:49:24.992529894Z ovnk_database_store/ovnkube-node-46t57_nbdb
2026-04-24T22:49:25.001633559Z ovnk_database_store/ovnkube-node-bbfmd_nbdb
2026-04-24T22:49:25.013267611Z INFO: Network log collection complete.
2026-04-24T22:49:25.070957157Z Wrote inspect data to must-gather.
2026-04-24T22:49:25.092531984Z INFO: Getting status from alertmanager-main-0
2026-04-24T22:49:25.338181543Z Wrote inspect data to must-gather.
2026-04-24T22:49:25.582029431Z W0424 22:49:25.581983    1436 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-24T22:49:25.588441492Z Wrote inspect data to must-gather.
2026-04-24T22:49:25.826463237Z Gathering data for ns/openshift-network-console...
2026-04-24T22:49:25.924377645Z Wrote inspect data to must-gather.
2026-04-24T22:49:26.217807259Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T22:49:26.221140853Z INFO: Fallback to identify the container image from release info
2026-04-24T22:49:26.302676145Z Gathering data for ns/openshift-console-operator...
2026-04-24T22:49:26.707327639Z Gathering data for ns/openshift-console...
2026-04-24T22:49:26.874492494Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T22:49:27.039812942Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T22:49:27.209288578Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T22:49:27.209394221Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T22:49:27.236602221Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T22:49:27.456740384Z Wrote inspect data to must-gather.
2026-04-24T22:49:27.596412335Z Gathering data for ns/openshift-dns-operator...
2026-04-24T22:49:27.895176179Z Gathering data for ns/openshift-dns...
2026-04-24T22:49:28.345708570Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T22:49:28.473080147Z Gathering data for ns/openshift-image-registry...
2026-04-24T22:49:28.907549668Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T22:49:29.216289292Z Gathering data for ns/openshift-ingress...
2026-04-24T22:49:29.471115159Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T22:49:29.702428248Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T22:49:30.125142348Z Gathering data for ns/openshift-insights...
2026-04-24T22:49:30.594575094Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T22:49:31.722812768Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T22:49:32.241131163Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T22:49:32.329202532Z Gathering data for ns/openshift-lws-operator...
2026-04-24T22:49:32.851663790Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T22:49:33.238589702Z Gathering data for ns/openshift-config...
2026-04-24T22:49:33.484319002Z Gathering data for ns/openshift-config-managed...
2026-04-24T22:49:33.922004127Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T22:49:33.994606891Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T22:49:34.202426540Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T22:49:34.527771288Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T22:49:34.832980643Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T22:49:35.116807844Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T22:49:35.140916684Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T22:49:35.402464910Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T22:49:35.434383088Z Collecting performance related data for node ip-10-0-133-9.ec2.internal
2026-04-24T22:49:35.443168608Z Collecting performance related data for node ip-10-0-129-176.ec2.internal
2026-04-24T22:49:35.454450065Z Collecting performance related data for node ip-10-0-134-101.ec2.internal
2026-04-24T22:49:35.746377306Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T22:49:36.153614278Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T22:49:36.551660744Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T22:49:37.206627238Z Gathering data for ns/openshift-multus...
2026-04-24T22:49:37.245566088Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T22:49:38.636152442Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T22:49:40.140337802Z Gathering data for ns/openshift-host-network...
2026-04-24T22:49:40.537679253Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T22:49:41.057602184Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T22:49:41.491809243Z Gathering data for ns/openshift-network-operator...
2026-04-24T22:49:41.834490635Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T22:49:42.095320342Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T22:49:42.249957461Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T22:49:42.603294701Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T22:49:42.971689417Z Gathering data for ns/openshift-apiserver...
2026-04-24T22:49:43.376082134Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T22:49:43.568777806Z Collecting kubelet logs for node ip-10-0-133-9.ec2.internal
2026-04-24T22:49:43.571456268Z Collecting kubelet logs for node ip-10-0-129-176.ec2.internal
2026-04-24T22:49:43.573657561Z Collecting kubelet logs for node ip-10-0-134-101.ec2.internal
2026-04-24T22:49:43.666215623Z Gathering data for ns/openshift-controller-manager...
2026-04-24T22:49:43.950898264Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T22:49:43.956313841Z INFO: Node performance data collection complete.
2026-04-24T22:49:43.992987906Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T22:49:44.626138724Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T22:49:44.924107165Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T22:49:45.244770287Z Gathering data for ns/openshift-service-ca...
2026-04-24T22:49:45.602697251Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T22:49:46.454283246Z Wrote inspect data to must-gather.
2026-04-24T22:49:46.454326802Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:49:46.454326802Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T22:49:46.582159374Z error: the server doesn't have a resource type "clusters"
2026-04-24T22:49:46.755438854Z Caches written to disk