2026-04-23T18:41:04.555894475Z [disk usage checker] Started
2026-04-23T18:41:04.589600335Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-23T18:41:04.774819853Z Gathering data for ns/openshift-cluster-version...
2026-04-23T18:41:04.916989366Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-23T18:41:05.011845385Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T18:41:05.115606025Z Gathering data for ns/default...
2026-04-23T18:41:05.431985847Z Gathering data for ns/openshift...
2026-04-23T18:41:05.903400891Z Gathering data for ns/kube-system...
2026-04-23T18:41:06.644383368Z Gathering data for ns/openshift-etcd...
2026-04-23T18:41:06.984069857Z Wrote inspect data to must-gather.
2026-04-23T18:41:06.984104772Z error: inspection completed with the errors occurred while gathering data:
2026-04-23T18:41:06.984104772Z     namespaces "assisted-installer" not found
2026-04-23T18:41:07.044721422Z Waiting on subprocesses to finish execution.
2026-04-23T18:41:07.058369616Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-23T18:41:07.060847459Z INFO: Gathering HAProxy config files
2026-04-23T18:41:07.083700889Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-23T18:41:07.110159386Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-23T18:41:07.112394320Z INFO: Collecting host service logs for crio
2026-04-23T18:41:07.112680167Z INFO: Collecting host service logs for kubelet
2026-04-23T18:41:07.112998821Z INFO: Collecting host service logs for rpm-ostreed
2026-04-23T18:41:07.113302787Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-23T18:41:07.113647480Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-23T18:41:07.114016629Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-23T18:41:07.114356472Z INFO: Collecting host service logs for NetworkManager
2026-04-23T18:41:07.114695286Z INFO: Collecting host service logs for openvswitch
2026-04-23T18:41:07.115039187Z INFO: Collecting host service logs for ovs-configuration
2026-04-23T18:41:07.115374725Z INFO: Collecting host service logs for ovsdb-server
2026-04-23T18:41:07.115729764Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-23T18:41:07.116153427Z INFO: Waiting for worker host service log collection to complete ...
2026-04-23T18:41:07.346866662Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-23T18:41:07.392291234Z INFO: Waiting for node performance related collection to complete ...
2026-04-23T18:41:07.435628417Z No resources found in openshift-etcd namespace.
2026-04-23T18:41:07.588478233Z No resources found
2026-04-23T18:41:07.869478573Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-23T18:41:07.979858592Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-23T18:41:08.056171161Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-23T18:41:08.056204112Z INFO: on-disk MachineConfig config collection complete.
2026-04-23T18:41:08.154945577Z INFO: "metallb-operator" not detected. Skipping.
2026-04-23T18:41:08.316099609Z Wrote inspect data to must-gather.
2026-04-23T18:41:08.453684379Z No resources found
2026-04-23T18:41:08.585314563Z error: the server doesn't have a resource type "performanceprofile"
2026-04-23T18:41:08.700539895Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-23T18:41:08.939258021Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-23T18:41:08.965611932Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-kfcjl
2026-04-23T18:41:08.965659680Z insights-runtime-extractor-64p9d
2026-04-23T18:41:08.965679414Z insights-runtime-extractor-jf562
2026-04-23T18:41:08.965698955Z insights-runtime-extractor-vn5xz
2026-04-23T18:41:08.972368140Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.036531113Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T18:41:09.047178199Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-23T18:41:09.053511864Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.096697160Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.146444049Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-23T18:41:09.173937470Z INFO: Worker host service log collection to complete.
2026-04-23T18:41:09.232037566Z ERROR: No running kube-apiserver pods found
2026-04-23T18:41:09.259627354Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.287505412Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-23T18:41:09.287505412Z See 'oc rsync -h' for help and examples
2026-04-23T18:41:09.328840798Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.397778095Z tar: Removing leading `/' from member names
2026-04-23T18:41:09.404111705Z INFO: HAProxy config collection complete.
2026-04-23T18:41:09.412838317Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.485068878Z INFO: 'previous-logs' folder not found on ip-10-0-131-177.ec2.internal, skipping...
2026-04-23T18:41:09.503198724Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.533913855Z error: the server doesn't have a resource type "machineconfigs"
2026-04-23T18:41:09.550614958Z Gathering data for ns/openshift-monitoring...
2026-04-23T18:41:09.595540794Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-23T18:41:09.665948058Z INFO: Getting rules from prometheus-k8s-0
2026-04-23T18:41:09.767786351Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.770011992Z Wrote inspect data to must-gather.
2026-04-23T18:41:09.778856105Z INFO: 'previous-logs' folder not found on ip-10-0-133-178.ec2.internal, skipping...
2026-04-23T18:41:10.024770305Z Wrote inspect data to must-gather.
2026-04-23T18:41:10.042226233Z INFO: 'previous-logs' folder not found on ip-10-0-136-172.ec2.internal, skipping...
2026-04-23T18:41:10.042259876Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-23T18:41:10.042259876Z INFO: Machine Config Daemon termination log collection complete.
2026-04-23T18:41:10.047896136Z Wrote inspect data to must-gather.
2026-04-23T18:41:10.097882352Z INFO: Getting status/config from prometheus-k8s-0
2026-04-23T18:41:10.191106234Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-23T18:41:10.339497493Z INFO: INTERCONNECT MODE
2026-04-23T18:41:10.339497493Z INFO: Gathering ovn-kubernetes DBs
2026-04-23T18:41:10.403765843Z Wrote inspect data to must-gather.
2026-04-23T18:41:10.506693878Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-23T18:41:10.514927291Z INFO: Gathering OVN_Northbound from ovnkube-node-7wdbp...
2026-04-23T18:41:10.515283461Z INFO: Gathering OVN_Northbound from ovnkube-node-dghxj...
2026-04-23T18:41:10.515538777Z INFO: Gathering OVN_Northbound from ovnkube-node-v9pcc...
2026-04-23T18:41:10.515899842Z INFO: Gathering OVN_Southbound from ovnkube-node-7wdbp...
2026-04-23T18:41:10.516212809Z INFO: Gathering OVN_Southbound from ovnkube-node-dghxj...
2026-04-23T18:41:10.516455193Z INFO: Gathering OVN_Southbound from ovnkube-node-v9pcc...
2026-04-23T18:41:10.724002177Z Wrote inspect data to must-gather.
2026-04-23T18:41:10.796192730Z tar: Removing leading `/' from member names
2026-04-23T18:41:10.801808822Z tar: Removing leading `/' from member names
2026-04-23T18:41:10.814040558Z tar: Removing leading `/' from member names
2026-04-23T18:41:10.837050565Z tar: Removing leading `/' from member names
2026-04-23T18:41:10.843345367Z tar: Removing leading `/' from member names
2026-04-23T18:41:10.848166806Z tar: Removing leading `/' from member names
2026-04-23T18:41:10.864249847Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-23T18:41:10.978737576Z Wrote inspect data to must-gather.
2026-04-23T18:41:11.132095823Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-23T18:41:11.206604654Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-23T18:41:11.209727893Z Wrote inspect data to must-gather.
2026-04-23T18:41:11.339024326Z INFO: Waiting for network log collection to complete ...
2026-04-23T18:41:11.339063725Z INFO: Waiting for ovnk database copies to complete ...
2026-04-23T18:41:11.339438367Z INFO: Copying ovnk databases complete.
2026-04-23T18:41:11.341053420Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-23T18:41:11.343632240Z ovnk_database_store/
2026-04-23T18:41:11.343632240Z ovnk_database_store/ovnkube-node-7wdbp_nbdb
2026-04-23T18:41:11.355100053Z ovnk_database_store/ovnkube-node-v9pcc_sbdb
2026-04-23T18:41:11.375438049Z ovnk_database_store/ovnkube-node-dghxj_nbdb
2026-04-23T18:41:11.385933727Z ovnk_database_store/ovnkube-node-7wdbp_sbdb
2026-04-23T18:41:11.409750916Z ovnk_database_store/ovnkube-node-v9pcc_nbdb
2026-04-23T18:41:11.410944538Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-23T18:41:11.418865918Z ovnk_database_store/ovnkube-node-dghxj_sbdb
2026-04-23T18:41:11.432754487Z Wrote inspect data to must-gather.
2026-04-23T18:41:11.444913422Z INFO: Network log collection complete.
2026-04-23T18:41:11.653573087Z Wrote inspect data to must-gather.
2026-04-23T18:41:11.825177842Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T18:41:11.847362598Z INFO: Getting status from alertmanager-main-0
2026-04-23T18:41:11.927497813Z Wrote inspect data to must-gather.
2026-04-23T18:41:11.997203970Z Gathering data for ns/openshift-network-console...
2026-04-23T18:41:12.197786535Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-23T18:41:12.200406585Z INFO: Fallback to identify the container image from release info
2026-04-23T18:41:12.437401872Z Gathering data for ns/openshift-console-operator...
2026-04-23T18:41:12.798864297Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-23T18:41:12.845075656Z Gathering data for ns/openshift-console...
2026-04-23T18:41:12.924793526Z daemonset.apps/perf-node-gather-daemonset created
2026-04-23T18:41:13.063160397Z Waiting for performance profile collector pods to become ready: 1
2026-04-23T18:41:13.336407637Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-23T18:41:13.693081567Z Gathering data for ns/openshift-dns-operator...
2026-04-23T18:41:13.967196261Z Gathering data for ns/openshift-dns...
2026-04-23T18:41:14.176695987Z Waiting for performance profile collector pods to become ready: 2
2026-04-23T18:41:14.578911531Z Gathering data for ns/openshift-image-registry...
2026-04-23T18:41:14.600196725Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-23T18:41:15.060484348Z Gathering data for ns/openshift-ingress-operator...
2026-04-23T18:41:15.311767969Z Waiting for performance profile collector pods to become ready: 3
2026-04-23T18:41:15.409899468Z Gathering data for ns/openshift-ingress...
2026-04-23T18:41:15.763354839Z Gathering data for ns/openshift-ingress-canary...
2026-04-23T18:41:16.159558267Z Gathering data for ns/openshift-insights...
2026-04-23T18:41:16.434882329Z Waiting for performance profile collector pods to become ready: 4
2026-04-23T18:41:17.581369701Z Waiting for performance profile collector pods to become ready: 5
2026-04-23T18:41:18.403271888Z Gathering data for ns/kserve...
2026-04-23T18:41:18.720491202Z Waiting for performance profile collector pods to become ready: 6
2026-04-23T18:41:19.606076792Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-23T18:41:19.837494761Z Waiting for performance profile collector pods to become ready: 7
2026-04-23T18:41:20.025375197Z Gathering data for ns/openshift-config...
2026-04-23T18:41:20.307725419Z Gathering data for ns/openshift-config-managed...
2026-04-23T18:41:20.776542394Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-23T18:41:20.963224804Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-23T18:41:21.071662657Z Gathering data for ns/openshift-kube-apiserver...
2026-04-23T18:41:21.248070752Z Collecting performance related data for node ip-10-0-136-172.ec2.internal
2026-04-23T18:41:21.255006231Z Collecting performance related data for node ip-10-0-131-177.ec2.internal
2026-04-23T18:41:21.263783030Z Collecting performance related data for node ip-10-0-133-178.ec2.internal
2026-04-23T18:41:21.419843488Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-23T18:41:21.816873479Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-23T18:41:22.214155797Z Gathering data for ns/openshift-kube-scheduler...
2026-04-23T18:41:22.568453277Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-23T18:41:22.955599735Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-23T18:41:23.295950675Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-23T18:41:23.628940099Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-23T18:41:24.244668950Z Gathering data for ns/openshift-multus...
2026-04-23T18:41:24.610550505Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-23T18:41:25.751431378Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-23T18:41:27.274861251Z Gathering data for ns/openshift-host-network...
2026-04-23T18:41:27.622099050Z Gathering data for ns/openshift-network-diagnostics...
2026-04-23T18:41:28.186660523Z Gathering data for ns/openshift-network-node-identity...
2026-04-23T18:41:28.620840945Z Gathering data for ns/openshift-network-operator...
2026-04-23T18:41:29.039828461Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-23T18:41:29.332554756Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-23T18:41:29.533569776Z Collecting kubelet logs for node ip-10-0-136-172.ec2.internal
2026-04-23T18:41:29.535372606Z Collecting kubelet logs for node ip-10-0-131-177.ec2.internal
2026-04-23T18:41:29.537020234Z Collecting kubelet logs for node ip-10-0-133-178.ec2.internal
2026-04-23T18:41:29.616928631Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-23T18:41:29.862715767Z Gathering data for ns/openshift-apiserver-operator...
2026-04-23T18:41:29.952560460Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-23T18:41:29.957655941Z INFO: Node performance data collection complete.
2026-04-23T18:41:30.199493286Z Gathering data for ns/openshift-apiserver...
2026-04-23T18:41:30.601505922Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-23T18:41:30.914338743Z Gathering data for ns/openshift-controller-manager...
2026-04-23T18:41:31.192654788Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-23T18:41:31.839690305Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-23T18:41:32.128127446Z Gathering data for ns/openshift-service-ca-operator...
2026-04-23T18:41:32.441687956Z Gathering data for ns/openshift-service-ca...
2026-04-23T18:41:32.799268771Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-23T18:41:33.692082806Z Wrote inspect data to must-gather.
2026-04-23T18:41:33.692131688Z error: inspection completed with the errors occurred while gathering data:
2026-04-23T18:41:33.692131688Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-23T18:41:33.830761833Z error: the server doesn't have a resource type "clusters"
2026-04-23T18:41:33.978949010Z Caches written to disk