2026-04-22T19:48:14.330461993Z [disk usage checker] Started
2026-04-22T19:48:14.403273172Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-22T19:48:14.556340637Z Gathering data for ns/openshift-cluster-version...
2026-04-22T19:48:14.696018195Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T19:48:14.801453054Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:48:14.886926975Z Gathering data for ns/default...
2026-04-22T19:48:15.240344002Z Gathering data for ns/openshift...
2026-04-22T19:48:15.697488462Z Gathering data for ns/kube-system...
2026-04-22T19:48:16.511716262Z Gathering data for ns/openshift-etcd...
2026-04-22T19:48:16.892092113Z Waiting on subprocesses to finish execution.
2026-04-22T19:48:16.898373591Z INFO: Collecting host service logs for crio
2026-04-22T19:48:16.898633529Z INFO: Gathering HAProxy config files
2026-04-22T19:48:16.898794533Z INFO: Collecting host service logs for kubelet
2026-04-22T19:48:16.899169945Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T19:48:16.899504660Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T19:48:16.899911602Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T19:48:16.900256324Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T19:48:16.900314058Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T19:48:16.900652643Z INFO: Collecting host service logs for NetworkManager
2026-04-22T19:48:16.901054575Z INFO: Collecting host service logs for openvswitch
2026-04-22T19:48:16.901409079Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T19:48:16.902955544Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T19:48:16.903436329Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T19:48:16.903858554Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T19:48:16.923797158Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T19:48:16.932802849Z Wrote inspect data to must-gather.
2026-04-22T19:48:16.932936754Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T19:48:16.932936754Z     namespaces "assisted-installer" not found
2026-04-22T19:48:16.935618705Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T19:48:16.994167695Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T19:48:17.912581996Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-g5jd8
2026-04-22T19:48:17.912639241Z insights-runtime-extractor-fn6js
2026-04-22T19:48:17.912660472Z insights-runtime-extractor-rq2j9
2026-04-22T19:48:17.912679307Z insights-runtime-extractor-x8ngq
2026-04-22T19:48:17.926633529Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T19:48:17.948326171Z No resources found
2026-04-22T19:48:17.962320323Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T19:48:18.006607594Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T19:48:18.046438629Z INFO: Worker host service log collection to complete.
2026-04-22T19:48:18.086118584Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T19:48:18.103165016Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T19:48:18.149066084Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T19:48:18.149860663Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T19:48:18.246390291Z Wrote inspect data to must-gather.
2026-04-22T19:48:18.255061516Z No resources found in openshift-etcd namespace.
2026-04-22T19:48:18.266174759Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T19:48:18.316101714Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T19:48:18.316101714Z See 'oc rsync -h' for help and examples
2026-04-22T19:48:18.341224954Z ERROR: No running kube-apiserver pods found
2026-04-22T19:48:18.539524870Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T19:48:18.610656966Z No resources found
2026-04-22T19:48:18.750946285Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T19:48:18.886520312Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:48:18.911344713Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T19:48:18.930813136Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.147277418Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.164216512Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.171134270Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.201817784Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.221062976Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.282105108Z INFO: 'previous-logs' folder not found on ip-10-0-131-132.ec2.internal, skipping...
2026-04-22T19:48:19.303117011Z tar: Removing leading `/' from member names
2026-04-22T19:48:19.309543733Z INFO: HAProxy config collection complete.
2026-04-22T19:48:19.417350690Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T19:48:19.417460783Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-22T19:48:19.471914340Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.524794940Z Gathering data for ns/openshift-monitoring...
2026-04-22T19:48:19.565192182Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T19:48:19.574810858Z INFO: 'previous-logs' folder not found on ip-10-0-133-159.ec2.internal, skipping...
2026-04-22T19:48:19.684965997Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.762242131Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.845836387Z INFO: 'previous-logs' folder not found on ip-10-0-138-15.ec2.internal, skipping...
2026-04-22T19:48:19.845947280Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T19:48:19.845998156Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T19:48:19.921677165Z Wrote inspect data to must-gather.
2026-04-22T19:48:19.986176172Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T19:48:20.023933188Z Wrote inspect data to must-gather.
2026-04-22T19:48:20.088257723Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T19:48:20.304566286Z Wrote inspect data to must-gather.
2026-04-22T19:48:20.322250084Z INFO: INTERCONNECT MODE
2026-04-22T19:48:20.322287275Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T19:48:20.391600133Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T19:48:20.508226138Z INFO: Gathering OVN_Northbound from ovnkube-node-2crp2...
2026-04-22T19:48:20.508483064Z INFO: Gathering OVN_Northbound from ovnkube-node-cszr4...
2026-04-22T19:48:20.508722488Z INFO: Gathering OVN_Northbound from ovnkube-node-w57k8...
2026-04-22T19:48:20.509013669Z INFO: Gathering OVN_Southbound from ovnkube-node-2crp2...
2026-04-22T19:48:20.509238005Z INFO: Gathering OVN_Southbound from ovnkube-node-cszr4...
2026-04-22T19:48:20.509469251Z INFO: Gathering OVN_Southbound from ovnkube-node-w57k8...
2026-04-22T19:48:20.534430799Z Wrote inspect data to must-gather.
2026-04-22T19:48:20.665316580Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T19:48:20.824917589Z tar: Removing leading `/' from member names
2026-04-22T19:48:20.827329193Z tar: Removing leading `/' from member names
2026-04-22T19:48:20.832409256Z tar: Removing leading `/' from member names
2026-04-22T19:48:20.837461166Z tar: Removing leading `/' from member names
2026-04-22T19:48:20.843840050Z Wrote inspect data to must-gather.
2026-04-22T19:48:20.856450700Z tar: Removing leading `/' from member names
2026-04-22T19:48:20.860109201Z tar: Removing leading `/' from member names
2026-04-22T19:48:20.983050214Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T19:48:21.085035987Z Wrote inspect data to must-gather.
2026-04-22T19:48:21.257693268Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T19:48:21.269915533Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T19:48:21.308756281Z INFO: Waiting for network log collection to complete ...
2026-04-22T19:48:21.308789157Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T19:48:21.309169434Z INFO: Copying ovnk databases complete.
2026-04-22T19:48:21.310685153Z 3.3M	must-gather/network_logs/ovnk_database_store
2026-04-22T19:48:21.312688295Z ovnk_database_store/
2026-04-22T19:48:21.312688295Z ovnk_database_store/ovnkube-node-2crp2_sbdb
2026-04-22T19:48:21.335353020Z Wrote inspect data to must-gather.
2026-04-22T19:48:21.336966734Z ovnk_database_store/ovnkube-node-cszr4_sbdb
2026-04-22T19:48:21.358964588Z ovnk_database_store/ovnkube-node-cszr4_nbdb
2026-04-22T19:48:21.368333108Z ovnk_database_store/ovnkube-node-w57k8_sbdb
2026-04-22T19:48:21.389603619Z ovnk_database_store/ovnkube-node-w57k8_nbdb
2026-04-22T19:48:21.397483118Z ovnk_database_store/ovnkube-node-2crp2_nbdb
2026-04-22T19:48:21.411956053Z INFO: Network log collection complete.
2026-04-22T19:48:21.548708505Z Wrote inspect data to must-gather.
2026-04-22T19:48:21.672985791Z INFO: Getting status from alertmanager-main-0
2026-04-22T19:48:21.767152165Z W0422 19:48:21.767100    1404 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-22T19:48:21.772469710Z Wrote inspect data to must-gather.
2026-04-22T19:48:21.883821411Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:48:22.045127264Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T19:48:22.045745197Z Gathering data for ns/openshift-network-console...
2026-04-22T19:48:22.047272808Z INFO: Fallback to identify the container image from release info
2026-04-22T19:48:22.458419746Z Gathering data for ns/openshift-console-operator...
2026-04-22T19:48:22.722160613Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T19:48:22.869695378Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T19:48:22.950494989Z Gathering data for ns/openshift-console...
2026-04-22T19:48:23.004336543Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T19:48:23.467338872Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T19:48:23.853171098Z Gathering data for ns/openshift-dns-operator...
2026-04-22T19:48:24.145698348Z Waiting for performance profile collector pods to become ready: 2
2026-04-22T19:48:24.174921913Z Gathering data for ns/openshift-dns...
2026-04-22T19:48:24.422408683Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-22T19:48:24.862723797Z Gathering data for ns/openshift-image-registry...
2026-04-22T19:48:25.287011733Z Waiting for performance profile collector pods to become ready: 3
2026-04-22T19:48:25.397621044Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T19:48:25.738900785Z Gathering data for ns/openshift-ingress...
2026-04-22T19:48:26.092645041Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T19:48:26.408807118Z Waiting for performance profile collector pods to become ready: 4
2026-04-22T19:48:26.597862354Z Gathering data for ns/openshift-insights...
2026-04-22T19:48:27.531422208Z Waiting for performance profile collector pods to become ready: 5
2026-04-22T19:48:28.665317406Z Waiting for performance profile collector pods to become ready: 6
2026-04-22T19:48:28.878517352Z Gathering data for ns/kserve...
2026-04-22T19:48:29.427007690Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-22T19:48:29.794091101Z Waiting for performance profile collector pods to become ready: 7
2026-04-22T19:48:30.491761023Z Gathering data for ns/openshift-config...
2026-04-22T19:48:30.896769051Z Gathering data for ns/openshift-config-managed...
2026-04-22T19:48:30.944719972Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T19:48:31.214342005Z Collecting performance related data for node ip-10-0-138-15.ec2.internal
2026-04-22T19:48:31.221622964Z Collecting performance related data for node ip-10-0-131-132.ec2.internal
2026-04-22T19:48:31.229337027Z Collecting performance related data for node ip-10-0-133-159.ec2.internal
2026-04-22T19:48:31.445803442Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T19:48:31.780077651Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T19:48:32.173214357Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T19:48:32.562266232Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T19:48:32.987548201Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T19:48:33.287050903Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T19:48:33.661315115Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T19:48:34.083459141Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T19:48:34.433057537Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-22T19:48:34.477108388Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T19:48:35.051859590Z Gathering data for ns/openshift-multus...
2026-04-22T19:48:36.768670597Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T19:48:38.455706355Z Gathering data for ns/openshift-host-network...
2026-04-22T19:48:38.844170805Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T19:48:39.435117704Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T19:48:39.438204616Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-22T19:48:39.839658107Z Collecting kubelet logs for node ip-10-0-138-15.ec2.internal
2026-04-22T19:48:39.841408791Z Collecting kubelet logs for node ip-10-0-131-132.ec2.internal
2026-04-22T19:48:39.843126613Z Collecting kubelet logs for node ip-10-0-133-159.ec2.internal
2026-04-22T19:48:39.878154913Z Gathering data for ns/openshift-network-operator...
2026-04-22T19:48:40.238598491Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T19:48:40.244950516Z INFO: Node performance data collection complete.
2026-04-22T19:48:40.355196679Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T19:48:40.683497418Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T19:48:41.255182329Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T19:48:41.605019522Z Gathering data for ns/openshift-apiserver...
2026-04-22T19:48:42.039335758Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T19:48:42.337867650Z Gathering data for ns/openshift-controller-manager...
2026-04-22T19:48:42.672428340Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T19:48:43.368583349Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T19:48:43.709299846Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T19:48:44.073282156Z Gathering data for ns/openshift-service-ca...
2026-04-22T19:48:44.442995296Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-22T19:48:44.489436382Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T19:48:45.479461837Z Wrote inspect data to must-gather.
2026-04-22T19:48:45.479508964Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T19:48:45.479508964Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T19:48:45.612100252Z error: the server doesn't have a resource type "clusters"
2026-04-22T19:48:45.754265070Z Caches written to disk