2026-06-09T07:47:38.734059011Z [disk usage checker] Started
2026-06-09T07:47:38.737951303Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-09T07:47:38.920310561Z Gathering data for ns/openshift-cluster-version...
2026-06-09T07:47:39.044746922Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T07:47:39.126576885Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T07:47:39.205403071Z Gathering data for ns/default...
2026-06-09T07:47:39.478888802Z Gathering data for ns/openshift...
2026-06-09T07:47:39.950519757Z Gathering data for ns/kube-system...
2026-06-09T07:47:40.630219815Z Gathering data for ns/openshift-etcd...
2026-06-09T07:47:40.919723903Z Wrote inspect data to must-gather.
2026-06-09T07:47:40.919759157Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T07:47:40.919759157Z     namespaces "assisted-installer" not found
2026-06-09T07:47:41.260240615Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-09T07:47:41.262689332Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-09T07:47:41.267848924Z Waiting on subprocesses to finish execution.
2026-06-09T07:47:41.269736914Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-09T07:47:41.269987812Z INFO: Gathering HAProxy config files
2026-06-09T07:47:41.287616610Z Executing Istio gather script
2026-06-09T07:47:41.309843091Z INFO: Collecting host service logs for crio
2026-06-09T07:47:41.310154775Z INFO: Collecting host service logs for kubelet
2026-06-09T07:47:41.310482833Z INFO: Collecting host service logs for rpm-ostreed
2026-06-09T07:47:41.310796098Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-09T07:47:41.311125564Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-09T07:47:41.311431838Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-09T07:47:41.311743759Z INFO: Collecting host service logs for NetworkManager
2026-06-09T07:47:41.312102487Z INFO: Collecting host service logs for openvswitch
2026-06-09T07:47:41.312507757Z INFO: Collecting host service logs for ovs-configuration
2026-06-09T07:47:41.312914668Z INFO: Collecting host service logs for ovsdb-server
2026-06-09T07:47:41.313247409Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-09T07:47:41.313632753Z INFO: Waiting for worker host service log collection to complete ...
2026-06-09T07:47:41.354080377Z INFO: Waiting for node performance related collection to complete ...
2026-06-09T07:47:41.794964436Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-09T07:47:41.795047527Z INFO: on-disk MachineConfig config collection complete.
2026-06-09T07:47:41.817849374Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-09T07:47:41.937524227Z INFO: "metallb-operator" not detected. Skipping.
2026-06-09T07:47:42.220009952Z error: the server doesn't have a resource type "performanceprofile"
2026-06-09T07:47:42.300579617Z INFO: Collecting Insights Archives from insights-operator-896549976-xvxdp
2026-06-09T07:47:42.300635159Z insights-runtime-extractor-6cw5d
2026-06-09T07:47:42.300655374Z insights-runtime-extractor-lbgrf
2026-06-09T07:47:42.300675478Z insights-runtime-extractor-s68bp
2026-06-09T07:47:42.331037307Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-09T07:47:42.499355544Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-09T07:47:42.544923739Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-09T07:47:42.602789868Z Wrote inspect data to must-gather.
2026-06-09T07:47:42.613692980Z No resources found
2026-06-09T07:47:42.744761344Z INFO: Worker host service log collection to complete.
2026-06-09T07:47:42.780503214Z ERROR: No running kube-apiserver pods found
2026-06-09T07:47:42.797952428Z No resources found in openshift-etcd namespace.
2026-06-09T07:47:43.005278433Z Wrote inspect data to must-gather.
2026-06-09T07:47:43.079523063Z Wrote inspect data to must-gather.
2026-06-09T07:47:43.291311323Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-09T07:47:43.291311323Z See 'oc rsync -h' for help and examples
2026-06-09T07:47:43.401204687Z Wrote inspect data to must-gather.
2026-06-09T07:47:43.424365529Z Gathering data for ns/openshift-monitoring...
2026-06-09T07:47:43.424839826Z No resources found
2026-06-09T07:47:43.491074950Z Wrote inspect data to must-gather.
2026-06-09T07:47:43.496701450Z error: the server doesn't have a resource type "machineconfigs"
2026-06-09T07:47:43.510398595Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-09T07:47:43.563515926Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T07:47:43.574506896Z Wrote inspect data to must-gather.
2026-06-09T07:47:43.630851067Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-09T07:47:43.672816963Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-09T07:47:43.779056456Z INFO: 'previous-logs' folder not found on ip-10-0-132-25.ec2.internal, skipping...
2026-06-09T07:47:43.779848687Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-09T07:47:43.846533854Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-09T07:47:43.927339149Z Wrote inspect data to must-gather.
2026-06-09T07:47:43.931327404Z tar: Removing leading `/' from member names
2026-06-09T07:47:43.935589710Z INFO: HAProxy config collection complete.
2026-06-09T07:47:44.092731082Z Wrote inspect data to must-gather.
2026-06-09T07:47:44.115634390Z Wrote inspect data to must-gather.
2026-06-09T07:47:44.147445296Z INFO: 'previous-logs' folder not found on ip-10-0-135-12.ec2.internal, skipping...
2026-06-09T07:47:44.208218477Z INFO: Getting rules from prometheus-k8s-0
2026-06-09T07:47:44.241780592Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-09T07:47:44.306239575Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-09T07:47:44.386826015Z Wrote inspect data to must-gather.
2026-06-09T07:47:44.435717553Z INFO: 'previous-logs' folder not found on ip-10-0-137-64.ec2.internal, skipping...
2026-06-09T07:47:44.435766574Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-09T07:47:44.435766574Z INFO: Machine Config Daemon termination log collection complete.
2026-06-09T07:47:44.529926712Z Wrote inspect data to must-gather.
2026-06-09T07:47:44.626388619Z INFO: Getting status/config from prometheus-k8s-0
2026-06-09T07:47:44.646829350Z Wrote inspect data to must-gather.
2026-06-09T07:47:44.747450525Z Wrote inspect data to must-gather.
2026-06-09T07:47:44.956247975Z INFO: INTERCONNECT MODE
2026-06-09T07:47:44.956345385Z INFO: Gathering ovn-kubernetes DBs
2026-06-09T07:47:44.974911882Z Wrote inspect data to must-gather.
2026-06-09T07:47:45.036016997Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-09T07:47:45.107769939Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T07:47:45.143725826Z INFO: Gathering OVN_Northbound from ovnkube-node-2x6lx...
2026-06-09T07:47:45.144021322Z INFO: Gathering OVN_Northbound from ovnkube-node-8gqll...
2026-06-09T07:47:45.144264332Z INFO: Gathering OVN_Northbound from ovnkube-node-kc5cq...
2026-06-09T07:47:45.144568684Z INFO: Gathering OVN_Southbound from ovnkube-node-2x6lx...
2026-06-09T07:47:45.144910027Z INFO: Gathering OVN_Southbound from ovnkube-node-8gqll...
2026-06-09T07:47:45.145209114Z INFO: Gathering OVN_Southbound from ovnkube-node-kc5cq...
2026-06-09T07:47:45.218551323Z Wrote inspect data to must-gather.
2026-06-09T07:47:45.296712400Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-09T07:47:45.313932513Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-09T07:47:45.440732814Z tar: Removing leading `/' from member names
2026-06-09T07:47:45.461140684Z tar: Removing leading `/' from member names
2026-06-09T07:47:45.501170424Z tar: Removing leading `/' from member names
2026-06-09T07:47:45.518082848Z tar: Removing leading `/' from member names
2026-06-09T07:47:45.540462150Z tar: Removing leading `/' from member names
2026-06-09T07:47:45.566600447Z tar: Removing leading `/' from member names
2026-06-09T07:47:45.584833765Z Wrote inspect data to must-gather.
2026-06-09T07:47:45.598085891Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:45.629898610Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-09T07:47:45.840409933Z W0609 07:47:45.840366    1403 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-09T07:47:45.845685709Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:45.847951068Z Wrote inspect data to must-gather.
2026-06-09T07:47:45.849850834Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-09T07:47:45.888205641Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T07:47:45.914600473Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-09T07:47:46.074608664Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:46.075434600Z INFO: Waiting for network log collection to complete ...
2026-06-09T07:47:46.075481192Z INFO: Waiting for ovnk database copies to complete ...
2026-06-09T07:47:46.075996714Z INFO: Copying ovnk databases complete.
2026-06-09T07:47:46.078097648Z 3.1M	must-gather/network_logs/ovnk_database_store
2026-06-09T07:47:46.081535225Z ovnk_database_store/
2026-06-09T07:47:46.081582375Z ovnk_database_store/ovnkube-node-8gqll_nbdb
2026-06-09T07:47:46.094604958Z ovnk_database_store/ovnkube-node-kc5cq_nbdb
2026-06-09T07:47:46.104841641Z ovnk_database_store/ovnkube-node-8gqll_sbdb
2026-06-09T07:47:46.112037357Z Gathering data for ns/openshift-network-console...
2026-06-09T07:47:46.121223760Z ovnk_database_store/ovnkube-node-2x6lx_sbdb
2026-06-09T07:47:46.137189085Z ovnk_database_store/ovnkube-node-2x6lx_nbdb
2026-06-09T07:47:46.145440822Z ovnk_database_store/ovnkube-node-kc5cq_sbdb
2026-06-09T07:47:46.169525036Z INFO: Network log collection complete.
2026-06-09T07:47:46.178740634Z Wrote inspect data to must-gather.
2026-06-09T07:47:46.309358356Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:46.314789772Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-09T07:47:46.328127145Z INFO: Getting status from alertmanager-main-0
2026-06-09T07:47:46.477410686Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-09T07:47:46.479988325Z INFO: Fallback to identify the container image from release info
2026-06-09T07:47:46.532225058Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:46.583043052Z Gathering data for ns/openshift-console-operator...
2026-06-09T07:47:46.730531716Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:46.732692192Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-09T07:47:46.938657702Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:46.945338854Z Gathering data for ns/openshift-console...
2026-06-09T07:47:47.118742097Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:47.121290130Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-09T07:47:47.226763365Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-09T07:47:47.298304832Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:47.364954062Z daemonset.apps/perf-node-gather-daemonset created
2026-06-09T07:47:47.493565120Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-09T07:47:47.508226603Z Waiting for performance profile collector pods to become ready: 1
2026-06-09T07:47:47.518431331Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:47:48.039429421Z Gathering data for ns/openshift-dns-operator...
2026-06-09T07:47:48.229736111Z Done executing Istio gather script
2026-06-09T07:47:48.323513516Z Gathering data for ns/openshift-dns...
2026-06-09T07:47:48.627517130Z Waiting for performance profile collector pods to become ready: 2
2026-06-09T07:47:48.786171791Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-09T07:47:48.874017361Z Gathering data for ns/openshift-image-registry...
2026-06-09T07:47:49.308730778Z Gathering data for ns/openshift-ingress-operator...
2026-06-09T07:47:49.638714061Z Gathering data for ns/openshift-ingress...
2026-06-09T07:47:49.750533776Z Waiting for performance profile collector pods to become ready: 3
2026-06-09T07:47:49.955892654Z Gathering data for ns/openshift-ingress-canary...
2026-06-09T07:47:50.353292398Z Gathering data for ns/openshift-insights...
2026-06-09T07:47:50.869467195Z Waiting for performance profile collector pods to become ready: 4
2026-06-09T07:47:51.977156496Z Waiting for performance profile collector pods to become ready: 5
2026-06-09T07:47:52.153349689Z Gathering data for ns/openshift-jobset-operator...
2026-06-09T07:47:52.911805906Z Gathering data for ns/openshift-config...
2026-06-09T07:47:53.089234198Z Waiting for performance profile collector pods to become ready: 6
2026-06-09T07:47:53.159631232Z Gathering data for ns/openshift-config-managed...
2026-06-09T07:47:53.591519686Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-09T07:47:53.792419233Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-09T07:47:53.881390545Z Gathering data for ns/openshift-kube-apiserver...
2026-06-09T07:47:54.115239965Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-09T07:47:54.216258184Z Waiting for performance profile collector pods to become ready: 7
2026-06-09T07:47:54.419256997Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-09T07:47:54.694390585Z Gathering data for ns/openshift-kube-scheduler...
2026-06-09T07:47:54.945754996Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-09T07:47:55.212706315Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-09T07:47:55.333849686Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-09T07:47:55.547959739Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-09T07:47:55.570461143Z Collecting performance related data for node ip-10-0-132-25.ec2.internal
2026-06-09T07:47:55.578202018Z Collecting performance related data for node ip-10-0-135-12.ec2.internal
2026-06-09T07:47:55.588897805Z Collecting performance related data for node ip-10-0-137-64.ec2.internal
2026-06-09T07:47:55.913025381Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-09T07:47:56.515702532Z Gathering data for ns/openshift-multus...
2026-06-09T07:47:58.006549863Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-09T07:47:58.797496047Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-09T07:47:59.260720951Z Gathering data for ns/openshift-host-network...
2026-06-09T07:47:59.559266340Z Gathering data for ns/openshift-network-diagnostics...
2026-06-09T07:48:00.079022292Z Gathering data for ns/openshift-network-node-identity...
2026-06-09T07:48:00.480115468Z Gathering data for ns/openshift-network-operator...
2026-06-09T07:48:00.855894869Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-09T07:48:01.120339882Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-09T07:48:01.601429637Z Gathering data for ns/openshift-apiserver-operator...
2026-06-09T07:48:01.912346797Z Gathering data for ns/openshift-apiserver...
2026-06-09T07:48:02.300927696Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-09T07:48:02.608741512Z Gathering data for ns/openshift-controller-manager...
2026-06-09T07:48:02.866024505Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-09T07:48:03.291505633Z Collecting kubelet logs for node ip-10-0-132-25.ec2.internal
2026-06-09T07:48:03.293545091Z Collecting kubelet logs for node ip-10-0-135-12.ec2.internal
2026-06-09T07:48:03.295893020Z Collecting kubelet logs for node ip-10-0-137-64.ec2.internal
2026-06-09T07:48:03.623739227Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-09T07:48:03.627923934Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vrtfb namespace
2026-06-09T07:48:03.636838541Z INFO: Node performance data collection complete.
2026-06-09T07:48:03.802490555Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-09T07:48:04.064501518Z Gathering data for ns/openshift-service-ca-operator...
2026-06-09T07:48:04.414590649Z Gathering data for ns/openshift-service-ca...
2026-06-09T07:48:04.841086844Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-09T07:48:05.872362563Z Wrote inspect data to must-gather.
2026-06-09T07:48:05.872406921Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T07:48:05.872406921Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-09T07:48:05.998203782Z error: the server doesn't have a resource type "clusters"
2026-06-09T07:48:06.125790875Z Caches written to disk