2026-05-26T01:50:59.628855925Z [disk usage checker] Started
2026-05-26T01:50:59.633477269Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-26T01:50:59.817851255Z Gathering data for ns/openshift-cluster-version...
2026-05-26T01:50:59.939006579Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T01:51:00.014061231Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T01:51:00.087708063Z Gathering data for ns/default...
2026-05-26T01:51:00.362137465Z Gathering data for ns/openshift...
2026-05-26T01:51:00.794435052Z Gathering data for ns/kube-system...
2026-05-26T01:51:01.446309911Z Gathering data for ns/openshift-etcd...
2026-05-26T01:51:01.704249030Z Wrote inspect data to must-gather.
2026-05-26T01:51:01.704283193Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T01:51:01.704283193Z     namespaces "assisted-installer" not found
2026-05-26T01:51:01.844295661Z Waiting on subprocesses to finish execution.
2026-05-26T01:51:01.853257904Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-26T01:51:01.870216094Z INFO: Collecting host service logs for crio
2026-05-26T01:51:01.870912911Z INFO: Collecting host service logs for kubelet
2026-05-26T01:51:01.871283805Z INFO: Collecting host service logs for rpm-ostreed
2026-05-26T01:51:01.871630183Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-26T01:51:01.871982781Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-26T01:51:01.872335576Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-26T01:51:01.872839906Z INFO: Collecting host service logs for NetworkManager
2026-05-26T01:51:01.873192101Z INFO: Collecting host service logs for openvswitch
2026-05-26T01:51:01.873525248Z INFO: Collecting host service logs for ovs-configuration
2026-05-26T01:51:01.873891921Z INFO: Collecting host service logs for ovsdb-server
2026-05-26T01:51:01.874001643Z INFO: Gathering HAProxy config files
2026-05-26T01:51:01.874270617Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-26T01:51:01.874737924Z INFO: Waiting for worker host service log collection to complete ...
2026-05-26T01:51:01.892461132Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-26T01:51:01.894505372Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-26T01:51:01.972227900Z INFO: Waiting for node performance related collection to complete ...
2026-05-26T01:51:02.349818149Z No resources found in openshift-etcd namespace.
2026-05-26T01:51:02.497179389Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-26T01:51:02.806299377Z INFO: "metallb-operator" not detected. Skipping.
2026-05-26T01:51:02.820721773Z error: the server doesn't have a resource type "performanceprofile"
2026-05-26T01:51:02.867742910Z Wrote inspect data to must-gather.
2026-05-26T01:51:03.074039897Z No resources found
2026-05-26T01:51:03.099595080Z INFO: Worker host service log collection to complete.
2026-05-26T01:51:03.179505356Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-26T01:51:03.179579544Z INFO: on-disk MachineConfig config collection complete.
2026-05-26T01:51:03.247067809Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-26T01:51:03.282451341Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-vp2rx
2026-05-26T01:51:03.282523659Z insights-runtime-extractor-btmwd
2026-05-26T01:51:03.282544483Z insights-runtime-extractor-jvsss
2026-05-26T01:51:03.282564652Z insights-runtime-extractor-v27vq
2026-05-26T01:51:03.370877976Z Wrote inspect data to must-gather.
2026-05-26T01:51:03.388726716Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-26T01:51:03.464663846Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-26T01:51:03.488979079Z No resources found
2026-05-26T01:51:03.522021961Z ERROR: No running kube-apiserver pods found
2026-05-26T01:51:03.536561952Z Gathering data for ns/openshift-monitoring...
2026-05-26T01:51:03.579095126Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-26T01:51:03.579095126Z See 'oc rsync -h' for help and examples
2026-05-26T01:51:03.634359274Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T01:51:03.702463866Z Wrote inspect data to must-gather.
2026-05-26T01:51:03.705500877Z Wrote inspect data to must-gather.
2026-05-26T01:51:03.820330058Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-26T01:51:03.847686305Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-26T01:51:03.950323508Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-26T01:51:04.008222282Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.030006965Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.050094061Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.065272821Z error: the server doesn't have a resource type "machineconfigs"
2026-05-26T01:51:04.109544007Z INFO: 'previous-logs' folder not found on ip-10-0-137-202.ec2.internal, skipping...
2026-05-26T01:51:04.144506062Z tar: Removing leading `/' from member names
2026-05-26T01:51:04.151693727Z INFO: HAProxy config collection complete.
2026-05-26T01:51:04.279033540Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.286361442Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.366754886Z INFO: 'previous-logs' folder not found on ip-10-0-140-41.ec2.internal, skipping...
2026-05-26T01:51:04.378779478Z INFO: Getting rules from prometheus-k8s-0
2026-05-26T01:51:04.516217775Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.537537588Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.627237470Z INFO: 'previous-logs' folder not found on ip-10-0-143-139.ec2.internal, skipping...
2026-05-26T01:51:04.627348333Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-26T01:51:04.627407587Z INFO: Machine Config Daemon termination log collection complete.
2026-05-26T01:51:04.638348763Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-26T01:51:04.664824849Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-26T01:51:04.763048163Z INFO: Getting status/config from prometheus-k8s-0
2026-05-26T01:51:04.775662565Z Wrote inspect data to must-gather.
2026-05-26T01:51:04.853117582Z Wrote inspect data to must-gather.
2026-05-26T01:51:05.036046612Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T01:51:05.056184734Z INFO: INTERCONNECT MODE
2026-05-26T01:51:05.056218614Z INFO: Gathering ovn-kubernetes DBs
2026-05-26T01:51:05.070138088Z Wrote inspect data to must-gather.
2026-05-26T01:51:05.136294130Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-26T01:51:05.258217738Z INFO: Gathering OVN_Northbound from ovnkube-node-g2584...
2026-05-26T01:51:05.258500999Z INFO: Gathering OVN_Northbound from ovnkube-node-pmsns...
2026-05-26T01:51:05.258740303Z INFO: Gathering OVN_Northbound from ovnkube-node-qdm2l...
2026-05-26T01:51:05.259066045Z INFO: Gathering OVN_Southbound from ovnkube-node-g2584...
2026-05-26T01:51:05.259369712Z INFO: Gathering OVN_Southbound from ovnkube-node-pmsns...
2026-05-26T01:51:05.259822216Z INFO: Gathering OVN_Southbound from ovnkube-node-qdm2l...
2026-05-26T01:51:05.280944893Z Wrote inspect data to must-gather.
2026-05-26T01:51:05.428570026Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-26T01:51:05.548047785Z tar: Removing leading `/' from member names
2026-05-26T01:51:05.558466509Z tar: Removing leading `/' from member names
2026-05-26T01:51:05.591138179Z tar: Removing leading `/' from member names
2026-05-26T01:51:05.593135386Z tar: Removing leading `/' from member names
2026-05-26T01:51:05.602548698Z tar: Removing leading `/' from member names
2026-05-26T01:51:05.609369491Z tar: Removing leading `/' from member names
2026-05-26T01:51:05.624263902Z W0526 01:51:05.624225    1245 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-26T01:51:05.631569435Z Wrote inspect data to must-gather.
2026-05-26T01:51:05.729083590Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-26T01:51:05.748409533Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T01:51:05.932426811Z Wrote inspect data to must-gather.
2026-05-26T01:51:05.938472237Z Gathering data for ns/openshift-network-console...
2026-05-26T01:51:05.981514306Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-26T01:51:06.046544574Z INFO: Waiting for network log collection to complete ...
2026-05-26T01:51:06.046668541Z INFO: Waiting for ovnk database copies to complete ...
2026-05-26T01:51:06.047215301Z INFO: Copying ovnk databases complete.
2026-05-26T01:51:06.050067802Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-05-26T01:51:06.053984236Z ovnk_database_store/
2026-05-26T01:51:06.053984236Z ovnk_database_store/ovnkube-node-g2584_sbdb
2026-05-26T01:51:06.079724492Z ovnk_database_store/ovnkube-node-pmsns_nbdb
2026-05-26T01:51:06.088643470Z ovnk_database_store/ovnkube-node-qdm2l_nbdb
2026-05-26T01:51:06.100396252Z ovnk_database_store/ovnkube-node-qdm2l_sbdb
2026-05-26T01:51:06.120938354Z ovnk_database_store/ovnkube-node-g2584_nbdb
2026-05-26T01:51:06.129112138Z ovnk_database_store/ovnkube-node-pmsns_sbdb
2026-05-26T01:51:06.152836088Z INFO: Network log collection complete.
2026-05-26T01:51:06.226641909Z Wrote inspect data to must-gather.
2026-05-26T01:51:06.350024280Z Gathering data for ns/openshift-console-operator...
2026-05-26T01:51:06.356269736Z INFO: Getting status from alertmanager-main-0
2026-05-26T01:51:06.499793165Z Wrote inspect data to must-gather.
2026-05-26T01:51:06.707670091Z Gathering data for ns/openshift-console...
2026-05-26T01:51:06.733738199Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-26T01:51:06.736030582Z INFO: Fallback to identify the container image from release info
2026-05-26T01:51:07.094693106Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-26T01:51:07.290921956Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-26T01:51:07.411977097Z daemonset.apps/perf-node-gather-daemonset created
2026-05-26T01:51:07.436587770Z Gathering data for ns/openshift-dns-operator...
2026-05-26T01:51:07.538023995Z Waiting for performance profile collector pods to become ready: 1
2026-05-26T01:51:07.693728535Z Gathering data for ns/openshift-dns...
2026-05-26T01:51:08.217812813Z Gathering data for ns/openshift-image-registry...
2026-05-26T01:51:08.623824353Z Gathering data for ns/openshift-ingress-operator...
2026-05-26T01:51:08.659874917Z Waiting for performance profile collector pods to become ready: 2
2026-05-26T01:51:08.951887821Z Gathering data for ns/openshift-ingress...
2026-05-26T01:51:09.229854790Z Gathering data for ns/openshift-ingress-canary...
2026-05-26T01:51:09.562829214Z Gathering data for ns/openshift-insights...
2026-05-26T01:51:09.643289041Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-26T01:51:09.758692856Z Waiting for performance profile collector pods to become ready: 3
2026-05-26T01:51:10.868020805Z Waiting for performance profile collector pods to become ready: 4
2026-05-26T01:51:11.406682290Z Gathering data for ns/openshift-config...
2026-05-26T01:51:11.670961002Z Gathering data for ns/openshift-config-managed...
2026-05-26T01:51:11.994595432Z Waiting for performance profile collector pods to become ready: 5
2026-05-26T01:51:12.060163197Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-26T01:51:12.325300824Z Gathering data for ns/openshift-kube-apiserver...
2026-05-26T01:51:12.538381497Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-26T01:51:12.777322266Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-26T01:51:13.019743438Z Gathering data for ns/openshift-kube-scheduler...
2026-05-26T01:51:13.114800489Z Waiting for performance profile collector pods to become ready: 6
2026-05-26T01:51:13.256916094Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-26T01:51:13.492571226Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-26T01:51:13.761040809Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-26T01:51:14.020661697Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-26T01:51:14.238709447Z Waiting for performance profile collector pods to become ready: 7
2026-05-26T01:51:14.520380286Z Gathering data for ns/openshift-multus...
2026-05-26T01:51:14.648227738Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-26T01:51:15.354965895Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-26T01:51:15.619651369Z Collecting performance related data for node ip-10-0-140-41.ec2.internal
2026-05-26T01:51:15.628228538Z Collecting performance related data for node ip-10-0-143-139.ec2.internal
2026-05-26T01:51:15.637094054Z Collecting performance related data for node ip-10-0-137-202.ec2.internal
2026-05-26T01:51:15.773908294Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-26T01:51:17.251937657Z Gathering data for ns/openshift-host-network...
2026-05-26T01:51:17.492912864Z Gathering data for ns/openshift-network-diagnostics...
2026-05-26T01:51:17.958584212Z Gathering data for ns/openshift-network-node-identity...
2026-05-26T01:51:18.353694384Z Gathering data for ns/openshift-network-operator...
2026-05-26T01:51:18.718168536Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-26T01:51:18.936263545Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-26T01:51:19.326693233Z Gathering data for ns/openshift-apiserver-operator...
2026-05-26T01:51:19.589979399Z Gathering data for ns/openshift-apiserver...
2026-05-26T01:51:19.653061732Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-26T01:51:19.935046670Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-26T01:51:20.159405275Z Gathering data for ns/openshift-controller-manager...
2026-05-26T01:51:20.404639273Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-26T01:51:21.025867845Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-26T01:51:21.274390858Z Gathering data for ns/openshift-service-ca-operator...
2026-05-26T01:51:21.550336802Z Gathering data for ns/openshift-service-ca...
2026-05-26T01:51:21.854892302Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-26T01:51:22.630384794Z Wrote inspect data to must-gather.
2026-05-26T01:51:22.630422854Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T01:51:22.630422854Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-26T01:51:23.420524702Z Collecting kubelet logs for node ip-10-0-140-41.ec2.internal
2026-05-26T01:51:23.422384319Z Collecting kubelet logs for node ip-10-0-143-139.ec2.internal
2026-05-26T01:51:23.424052823Z Collecting kubelet logs for node ip-10-0-137-202.ec2.internal
2026-05-26T01:51:23.732448342Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-26T01:51:23.736741128Z INFO: Node performance data collection complete.
2026-05-26T01:51:23.861356722Z error: the server doesn't have a resource type "clusters"
2026-05-26T01:51:24.013016311Z Caches written to disk