2026-04-22T14:47:28.077556984Z [disk usage checker] Started
2026-04-22T14:47:28.116341609Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-04-22T14:47:28.446854783Z Gathering data for ns/openshift-cluster-version...
2026-04-22T14:47:28.588641606Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T14:47:28.680054996Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T14:47:28.774618388Z Gathering data for ns/default...
2026-04-22T14:47:29.111879739Z Gathering data for ns/openshift...
2026-04-22T14:47:29.575806696Z Gathering data for ns/kube-system...
2026-04-22T14:47:30.281514800Z Gathering data for ns/openshift-etcd...
2026-04-22T14:47:30.609332233Z Wrote inspect data to must-gather.
2026-04-22T14:47:30.609366900Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T14:47:30.609366900Z     namespaces "assisted-installer" not found
2026-04-22T14:47:30.868812024Z Waiting on subprocesses to finish execution.
2026-04-22T14:47:30.880251793Z INFO: Collecting host service logs for crio
2026-04-22T14:47:30.880596709Z INFO: Collecting host service logs for kubelet
2026-04-22T14:47:30.880745450Z INFO: Gathering HAProxy config files
2026-04-22T14:47:30.880983527Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T14:47:30.881362901Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T14:47:30.881694805Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T14:47:30.882009210Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T14:47:30.882103504Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T14:47:30.882413627Z INFO: Collecting host service logs for NetworkManager
2026-04-22T14:47:30.882754138Z INFO: Collecting host service logs for openvswitch
2026-04-22T14:47:30.883088482Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T14:47:30.883447774Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T14:47:30.883799580Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T14:47:30.884254637Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T14:47:30.884273075Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T14:47:30.888045510Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T14:47:30.949596947Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T14:47:31.577670216Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-zkfsf
2026-04-22T14:47:31.577670216Z insights-runtime-extractor-2k8nd
2026-04-22T14:47:31.577670216Z insights-runtime-extractor-dtswc
2026-04-22T14:47:31.577670216Z insights-runtime-extractor-j76jp
2026-04-22T14:47:31.724061500Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T14:47:31.724186389Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T14:47:32.056069611Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T14:47:32.120205147Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T14:47:32.125365624Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T14:47:32.252952084Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T14:47:32.253178790Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T14:47:32.253178790Z See 'oc rsync -h' for help and examples
2026-04-22T14:47:32.315458358Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T14:47:32.358308764Z INFO: Worker host service log collection to complete.
2026-04-22T14:47:32.372781143Z No resources found in openshift-etcd namespace.
2026-04-22T14:47:32.386895369Z Wrote inspect data to must-gather.
2026-04-22T14:47:32.435502663Z No resources found
2026-04-22T14:47:32.740725526Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T14:47:32.872568962Z ERROR: No running kube-apiserver pods found
2026-04-22T14:47:33.110166438Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T14:47:33.257759952Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-04-22T14:47:33.347835820Z No resources found
2026-04-22T14:47:33.396594202Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T14:47:33.477826486Z Wrote inspect data to must-gather.
2026-04-22T14:47:33.485675894Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T14:47:33.617901955Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T14:47:33.643528507Z tar: Removing leading `/' from member names
2026-04-22T14:47:33.650546956Z INFO: HAProxy config collection complete.
2026-04-22T14:47:33.669535471Z Wrote inspect data to must-gather.
2026-04-22T14:47:33.684810690Z Wrote inspect data to must-gather.
2026-04-22T14:47:33.699265607Z Wrote inspect data to must-gather.
2026-04-22T14:47:33.707537385Z INFO: 'previous-logs' folder not found on ip-10-0-132-130.ec2.internal, skipping...
2026-04-22T14:47:33.739411560Z Wrote inspect data to must-gather.
2026-04-22T14:47:33.859226287Z Gathering data for ns/kuadrant-system...
2026-04-22T14:47:33.932769928Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T14:47:34.116036248Z INFO: 'previous-logs' folder not found on ip-10-0-133-31.ec2.internal, skipping...
2026-04-22T14:47:34.130081714Z Wrote inspect data to must-gather.
2026-04-22T14:47:34.147161529Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T14:47:34.306394633Z Wrote inspect data to must-gather.
2026-04-22T14:47:34.435621508Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T14:47:34.519732620Z INFO: 'previous-logs' folder not found on ip-10-0-136-18.ec2.internal, skipping...
2026-04-22T14:47:34.519768468Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T14:47:34.519801524Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T14:47:34.564003672Z Wrote inspect data to must-gather.
2026-04-22T14:47:34.691160700Z Wrote inspect data to must-gather.
2026-04-22T14:47:34.725256012Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T14:47:34.755645727Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T14:47:34.950260124Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T14:47:34.995783478Z Wrote inspect data to must-gather.
2026-04-22T14:47:35.004676329Z Gathering data for ns/openshift-monitoring...
2026-04-22T14:47:35.249492433Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T14:47:35.313971222Z Wrote inspect data to must-gather.
2026-04-22T14:47:35.389879271Z INFO: INTERCONNECT MODE
2026-04-22T14:47:35.389963691Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T14:47:35.507325470Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T14:47:35.616504136Z Wrote inspect data to must-gather.
2026-04-22T14:47:35.618585104Z INFO: Gathering OVN_Northbound from ovnkube-node-2kpzl...
2026-04-22T14:47:35.618901261Z INFO: Gathering OVN_Northbound from ovnkube-node-gpjz8...
2026-04-22T14:47:35.619199448Z INFO: Gathering OVN_Northbound from ovnkube-node-k777w...
2026-04-22T14:47:35.619528848Z INFO: Gathering OVN_Southbound from ovnkube-node-2kpzl...
2026-04-22T14:47:35.619804653Z INFO: Gathering OVN_Southbound from ovnkube-node-gpjz8...
2026-04-22T14:47:35.620074720Z INFO: Gathering OVN_Southbound from ovnkube-node-k777w...
2026-04-22T14:47:35.871394956Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T14:47:35.958919402Z tar: Removing leading `/' from member names
2026-04-22T14:47:35.989842767Z tar: Removing leading `/' from member names
2026-04-22T14:47:36.028954615Z tar: Removing leading `/' from member names
2026-04-22T14:47:36.034637932Z tar: Removing leading `/' from member names
2026-04-22T14:47:36.066481107Z tar: Removing leading `/' from member names
2026-04-22T14:47:36.104467045Z tar: Removing leading `/' from member names
2026-04-22T14:47:36.164193117Z Wrote inspect data to must-gather.
2026-04-22T14:47:36.253197351Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T14:47:36.533652311Z Wrote inspect data to must-gather.
2026-04-22T14:47:36.749702526Z INFO: Waiting for network log collection to complete ...
2026-04-22T14:47:36.750075392Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T14:47:36.750275249Z INFO: Copying ovnk databases complete.
2026-04-22T14:47:36.755437596Z 4.8M	must-gather/network_logs/ovnk_database_store
2026-04-22T14:47:36.760161972Z ovnk_database_store/
2026-04-22T14:47:36.760226117Z ovnk_database_store/ovnkube-node-2kpzl_sbdb
2026-04-22T14:47:36.789935382Z INFO: Getting status from alertmanager-main-0
2026-04-22T14:47:36.797210324Z ovnk_database_store/ovnkube-node-2kpzl_nbdb
2026-04-22T14:47:36.809649413Z ovnk_database_store/ovnkube-node-k777w_nbdb
2026-04-22T14:47:36.823894498Z ovnk_database_store/ovnkube-node-k777w_sbdb
2026-04-22T14:47:36.858837778Z ovnk_database_store/ovnkube-node-gpjz8_nbdb
2026-04-22T14:47:36.862776721Z Wrote inspect data to must-gather.
2026-04-22T14:47:36.871570362Z ovnk_database_store/ovnkube-node-gpjz8_sbdb
2026-04-22T14:47:36.902244718Z INFO: Network log collection complete.
2026-04-22T14:47:37.211680547Z Wrote inspect data to must-gather.
2026-04-22T14:47:37.336168724Z Gathering data for ns/openshift-network-console...
2026-04-22T14:47:37.567881566Z Wrote inspect data to must-gather.
2026-04-22T14:47:37.866677493Z Gathering data for ns/openshift-console-operator...
2026-04-22T14:47:37.875226805Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T14:47:37.878358482Z INFO: Fallback to identify the container image from release info
2026-04-22T14:47:38.264203276Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-04-22T14:47:38.323420370Z Gathering data for ns/openshift-console...
2026-04-22T14:47:38.569245178Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T14:47:38.657007656Z Wrote inspect data to must-gather.
2026-04-22T14:47:38.715549809Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T14:47:38.861384752Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T14:47:38.870988127Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T14:47:39.239037562Z Gathering data for ns/openshift-dns-operator...
2026-04-22T14:47:39.559748818Z Gathering data for ns/openshift-dns...
2026-04-22T14:47:40.001506098Z Waiting for performance profile collector pods to become ready: 2
2026-04-22T14:47:40.232205596Z Gathering data for ns/openshift-image-registry...
2026-04-22T14:47:40.739896640Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T14:47:41.109663332Z Gathering data for ns/openshift-ingress...
2026-04-22T14:47:41.146531636Z Waiting for performance profile collector pods to become ready: 3
2026-04-22T14:47:41.661937712Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T14:47:42.096997539Z Gathering data for ns/openshift-insights...
2026-04-22T14:47:42.260548740Z Waiting for performance profile collector pods to become ready: 4
2026-04-22T14:47:43.269234568Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-04-22T14:47:43.375821845Z Waiting for performance profile collector pods to become ready: 5
2026-04-22T14:47:44.502150846Z Waiting for performance profile collector pods to become ready: 6
2026-04-22T14:47:44.797798801Z Gathering data for ns/openshift-lws-operator...
2026-04-22T14:47:45.343517625Z Gathering data for ns/kserve...
2026-04-22T14:47:45.627761821Z Waiting for performance profile collector pods to become ready: 7
2026-04-22T14:47:46.753912063Z Waiting for performance profile collector pods to become ready: 8
2026-04-22T14:47:47.123404557Z Gathering data for ns/openshift-config...
2026-04-22T14:47:47.496784020Z Gathering data for ns/openshift-config-managed...
2026-04-22T14:47:47.896292374Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T14:47:47.975502048Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T14:47:48.250561213Z Collecting performance related data for node ip-10-0-133-31.ec2.internal
2026-04-22T14:47:48.264243804Z Collecting performance related data for node ip-10-0-132-130.ec2.internal
2026-04-22T14:47:48.279255598Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-04-22T14:47:48.284132241Z Collecting performance related data for node ip-10-0-136-18.ec2.internal
2026-04-22T14:47:48.430688927Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T14:47:48.938866166Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T14:47:49.388900814Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T14:47:49.907762689Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T14:47:50.353192471Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T14:47:50.687607144Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T14:47:51.065853519Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T14:47:51.441148773Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T14:47:52.023731499Z Gathering data for ns/openshift-multus...
2026-04-22T14:47:53.290006953Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-04-22T14:47:53.536635267Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T14:47:55.387643469Z Gathering data for ns/openshift-host-network...
2026-04-22T14:47:55.758333336Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T14:47:56.313231846Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T14:47:56.798187092Z Gathering data for ns/openshift-network-operator...
2026-04-22T14:47:56.924785987Z Collecting kubelet logs for node ip-10-0-133-31.ec2.internal
2026-04-22T14:47:56.926993985Z Collecting kubelet logs for node ip-10-0-132-130.ec2.internal
2026-04-22T14:47:56.929330622Z Collecting kubelet logs for node ip-10-0-136-18.ec2.internal
2026-04-22T14:47:57.298730478Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T14:47:57.372664875Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T14:47:57.379771445Z INFO: Node performance data collection complete.
2026-04-22T14:47:57.623794633Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T14:47:58.096762184Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T14:47:58.297529178Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-04-22T14:47:58.423322582Z Gathering data for ns/openshift-apiserver...
2026-04-22T14:47:58.836238911Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T14:47:59.131960706Z Gathering data for ns/openshift-controller-manager...
2026-04-22T14:47:59.408800429Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T14:48:00.081538630Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T14:48:00.376400898Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T14:48:00.703238342Z Gathering data for ns/openshift-service-ca...
2026-04-22T14:48:01.083467417Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T14:48:02.023847250Z Wrote inspect data to must-gather.
2026-04-22T14:48:02.023897087Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T14:48:02.023897087Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T14:48:02.169645945Z error: the server doesn't have a resource type "clusters"
2026-04-22T14:48:02.349494836Z Caches written to disk