2026-06-05T11:51:59.622558028Z [disk usage checker] Started
2026-06-05T11:51:59.626372528Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-05T11:51:59.811396343Z Gathering data for ns/openshift-cluster-version...
2026-06-05T11:51:59.947102594Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T11:52:00.025589204Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T11:52:00.101929223Z Gathering data for ns/default...
2026-06-05T11:52:00.371651253Z Gathering data for ns/openshift...
2026-06-05T11:52:00.762562065Z Gathering data for ns/kube-system...
2026-06-05T11:52:01.383500944Z Gathering data for ns/openshift-etcd...
2026-06-05T11:52:01.650877566Z Wrote inspect data to must-gather.
2026-06-05T11:52:01.650985839Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T11:52:01.650985839Z     namespaces "assisted-installer" not found
2026-06-05T11:52:02.173829930Z Waiting on subprocesses to finish execution.
2026-06-05T11:52:02.184755891Z INFO: Gathering HAProxy config files
2026-06-05T11:52:02.190144332Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-05T11:52:02.195733635Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-05T11:52:02.195964536Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-05T11:52:02.200787754Z INFO: Collecting host service logs for crio
2026-06-05T11:52:02.201145552Z INFO: Collecting host service logs for kubelet
2026-06-05T11:52:02.201267032Z Executing Istio gather script
2026-06-05T11:52:02.203039123Z INFO: Collecting host service logs for rpm-ostreed
2026-06-05T11:52:02.203337681Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-05T11:52:02.203633498Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-05T11:52:02.203931173Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-05T11:52:02.204251356Z INFO: Collecting host service logs for NetworkManager
2026-06-05T11:52:02.204666644Z INFO: Collecting host service logs for openvswitch
2026-06-05T11:52:02.205023688Z INFO: Collecting host service logs for ovs-configuration
2026-06-05T11:52:02.205325438Z INFO: Collecting host service logs for ovsdb-server
2026-06-05T11:52:02.205627340Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-05T11:52:02.206000968Z INFO: Waiting for worker host service log collection to complete ...
2026-06-05T11:52:02.536881530Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-05T11:52:02.668262370Z INFO: Waiting for node performance related collection to complete ...
2026-06-05T11:52:02.728353826Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-05T11:52:02.894337159Z INFO: "metallb-operator" not detected. Skipping.
2026-06-05T11:52:02.916138492Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-05T11:52:02.949791054Z Wrote inspect data to must-gather.
2026-06-05T11:52:02.995426832Z INFO: Collecting Insights Archives from insights-operator-896549976-nwlz8
2026-06-05T11:52:02.995483132Z insights-runtime-extractor-88shg
2026-06-05T11:52:02.995503219Z insights-runtime-extractor-mg49b
2026-06-05T11:52:02.995522513Z insights-runtime-extractor-sst8b
2026-06-05T11:52:03.295874726Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-05T11:52:03.295967972Z INFO: on-disk MachineConfig config collection complete.
2026-06-05T11:52:03.347437491Z ERROR: No running kube-apiserver pods found
2026-06-05T11:52:03.424743793Z error: the server doesn't have a resource type "performanceprofile"
2026-06-05T11:52:03.509488352Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-05T11:52:03.509488352Z See 'oc rsync -h' for help and examples
2026-06-05T11:52:03.530663321Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-05T11:52:03.535238471Z No resources found
2026-06-05T11:52:03.643567691Z INFO: Worker host service log collection to complete.
2026-06-05T11:52:03.649267367Z No resources found in openshift-etcd namespace.
2026-06-05T11:52:03.882759195Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-05T11:52:03.887743505Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T11:52:04.198830550Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-05T11:52:04.222938730Z Wrote inspect data to must-gather.
2026-06-05T11:52:04.400527759Z Wrote inspect data to must-gather.
2026-06-05T11:52:04.423382525Z Wrote inspect data to must-gather.
2026-06-05T11:52:04.489500531Z No resources found
2026-06-05T11:52:04.504275615Z Wrote inspect data to must-gather.
2026-06-05T11:52:04.514599768Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-05T11:52:04.558053813Z Wrote inspect data to must-gather.
2026-06-05T11:52:04.635975856Z Wrote inspect data to must-gather.
2026-06-05T11:52:04.639064858Z INFO: 'previous-logs' folder not found on ip-10-0-129-98.ec2.internal, skipping...
2026-06-05T11:52:04.640140185Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-05T11:52:04.819779526Z Gathering data for ns/openshift-monitoring...
2026-06-05T11:52:04.854152824Z tar: Removing leading `/' from member names
2026-06-05T11:52:04.873383103Z Wrote inspect data to must-gather.
2026-06-05T11:52:04.877730019Z INFO: HAProxy config collection complete.
2026-06-05T11:52:04.878769466Z error: the server doesn't have a resource type "machineconfigs"
2026-06-05T11:52:04.958151087Z INFO: Getting rules from prometheus-k8s-0
2026-06-05T11:52:05.027014766Z INFO: 'previous-logs' folder not found on ip-10-0-135-216.ec2.internal, skipping...
2026-06-05T11:52:05.035872007Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-05T11:52:05.080988906Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-05T11:52:05.188061419Z Wrote inspect data to must-gather.
2026-06-05T11:52:05.337243838Z Wrote inspect data to must-gather.
2026-06-05T11:52:05.341565136Z INFO: 'previous-logs' folder not found on ip-10-0-140-113.ec2.internal, skipping...
2026-06-05T11:52:05.341594686Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-05T11:52:05.341594686Z INFO: Machine Config Daemon termination log collection complete.
2026-06-05T11:52:05.423072431Z INFO: Getting status/config from prometheus-k8s-0
2026-06-05T11:52:05.475536665Z Wrote inspect data to must-gather.
2026-06-05T11:52:05.506421387Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-05T11:52:05.713120273Z Wrote inspect data to must-gather.
2026-06-05T11:52:05.774683856Z INFO: INTERCONNECT MODE
2026-06-05T11:52:05.774723739Z INFO: Gathering ovn-kubernetes DBs
2026-06-05T11:52:05.829703671Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-05T11:52:05.947585601Z Wrote inspect data to must-gather.
2026-06-05T11:52:05.954192692Z INFO: Gathering OVN_Northbound from ovnkube-node-4dqcp...
2026-06-05T11:52:05.954401047Z INFO: Gathering OVN_Northbound from ovnkube-node-fxspx...
2026-06-05T11:52:05.954584019Z INFO: Gathering OVN_Northbound from ovnkube-node-vhmkh...
2026-06-05T11:52:05.954847487Z INFO: Gathering OVN_Southbound from ovnkube-node-4dqcp...
2026-06-05T11:52:05.955202449Z INFO: Gathering OVN_Southbound from ovnkube-node-fxspx...
2026-06-05T11:52:05.955499426Z INFO: Gathering OVN_Southbound from ovnkube-node-vhmkh...
2026-06-05T11:52:06.098100430Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-05T11:52:06.301322954Z tar: Removing leading `/' from member names
2026-06-05T11:52:06.304150183Z tar: Removing leading `/' from member names
2026-06-05T11:52:06.306240919Z tar: Removing leading `/' from member names
2026-06-05T11:52:06.313744118Z tar: Removing leading `/' from member names
2026-06-05T11:52:06.319015402Z tar: Removing leading `/' from member names
2026-06-05T11:52:06.325841955Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-05T11:52:06.329371614Z tar: Removing leading `/' from member names
2026-06-05T11:52:06.350674609Z Wrote inspect data to must-gather.
2026-06-05T11:52:06.398074408Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T11:52:06.481349420Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-05T11:52:06.532168989Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:06.587372455Z Wrote inspect data to must-gather.
2026-06-05T11:52:06.756973660Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-05T11:52:06.777325159Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:06.780148222Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-05T11:52:06.832201638Z Wrote inspect data to must-gather.
2026-06-05T11:52:06.878554061Z INFO: Waiting for network log collection to complete ...
2026-06-05T11:52:06.878672056Z INFO: Waiting for ovnk database copies to complete ...
2026-06-05T11:52:06.879241880Z INFO: Copying ovnk databases complete.
2026-06-05T11:52:06.881345229Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-06-05T11:52:06.884286900Z ovnk_database_store/
2026-06-05T11:52:06.884335892Z ovnk_database_store/ovnkube-node-4dqcp_sbdb
2026-06-05T11:52:06.907742326Z ovnk_database_store/ovnkube-node-4dqcp_nbdb
2026-06-05T11:52:06.915574778Z ovnk_database_store/ovnkube-node-vhmkh_nbdb
2026-06-05T11:52:06.922155718Z ovnk_database_store/ovnkube-node-fxspx_sbdb
2026-06-05T11:52:06.940001057Z ovnk_database_store/ovnkube-node-fxspx_nbdb
2026-06-05T11:52:06.947577534Z ovnk_database_store/ovnkube-node-vhmkh_sbdb
2026-06-05T11:52:06.967499036Z INFO: Network log collection complete.
2026-06-05T11:52:06.995190847Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:07.048152425Z W0605 11:52:07.048107    1490 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-05T11:52:07.052963179Z Wrote inspect data to must-gather.
2026-06-05T11:52:07.074569777Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T11:52:07.129627797Z INFO: Getting status from alertmanager-main-0
2026-06-05T11:52:07.220701810Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:07.222996714Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-05T11:52:07.278428976Z Gathering data for ns/openshift-network-console...
2026-06-05T11:52:07.379345273Z Wrote inspect data to must-gather.
2026-06-05T11:52:07.426095745Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:07.625880970Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:07.629233484Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-05T11:52:07.644553930Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-05T11:52:07.646474994Z INFO: Fallback to identify the container image from release info
2026-06-05T11:52:07.697341268Z Gathering data for ns/openshift-console-operator...
2026-06-05T11:52:07.853569185Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:08.029637034Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:08.031605986Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-05T11:52:08.055148403Z Gathering data for ns/openshift-console...
2026-06-05T11:52:08.197868370Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:08.386411127Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:52:08.428346881Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-05T11:52:08.567201214Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-05T11:52:08.572011319Z daemonset.apps/perf-node-gather-daemonset created
2026-06-05T11:52:08.713199453Z Waiting for performance profile collector pods to become ready: 1
2026-06-05T11:52:09.141649853Z Gathering data for ns/openshift-dns-operator...
2026-06-05T11:52:09.150529726Z Done executing Istio gather script
2026-06-05T11:52:09.438304532Z Gathering data for ns/openshift-dns...
2026-06-05T11:52:09.646982396Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-05T11:52:09.822453612Z Waiting for performance profile collector pods to become ready: 2
2026-06-05T11:52:09.968329999Z Gathering data for ns/openshift-image-registry...
2026-06-05T11:52:10.361502609Z Gathering data for ns/openshift-ingress-operator...
2026-06-05T11:52:10.714980810Z Gathering data for ns/openshift-ingress...
2026-06-05T11:52:10.937094977Z Waiting for performance profile collector pods to become ready: 3
2026-06-05T11:52:11.023686435Z Gathering data for ns/openshift-ingress-canary...
2026-06-05T11:52:11.355164461Z Gathering data for ns/openshift-insights...
2026-06-05T11:52:12.044138028Z Waiting for performance profile collector pods to become ready: 4
2026-06-05T11:52:13.067866408Z Gathering data for ns/openshift-jobset-operator...
2026-06-05T11:52:13.151598312Z Waiting for performance profile collector pods to become ready: 5
2026-06-05T11:52:13.809129925Z Gathering data for ns/openshift-config...
2026-06-05T11:52:14.034433803Z Gathering data for ns/openshift-config-managed...
2026-06-05T11:52:14.257793418Z Waiting for performance profile collector pods to become ready: 6
2026-06-05T11:52:14.417354788Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-05T11:52:14.634496947Z Gathering data for ns/openshift-kube-apiserver...
2026-06-05T11:52:14.653945557Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-05T11:52:14.862521109Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-05T11:52:15.114227625Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-05T11:52:15.380336171Z Gathering data for ns/openshift-kube-scheduler...
2026-06-05T11:52:15.383566291Z Waiting for performance profile collector pods to become ready: 7
2026-06-05T11:52:15.649836485Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-05T11:52:15.889143620Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-05T11:52:16.167398876Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-05T11:52:16.434185623Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-05T11:52:16.514486979Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-05T11:52:16.764123982Z Collecting performance related data for node ip-10-0-140-113.ec2.internal
2026-06-05T11:52:16.770786733Z Collecting performance related data for node ip-10-0-129-98.ec2.internal
2026-06-05T11:52:16.778527808Z Collecting performance related data for node ip-10-0-135-216.ec2.internal
2026-06-05T11:52:16.970545990Z Gathering data for ns/openshift-multus...
2026-06-05T11:52:18.355889554Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-05T11:52:19.538594250Z Gathering data for ns/openshift-host-network...
2026-06-05T11:52:19.659098433Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-05T11:52:19.867358827Z Gathering data for ns/openshift-network-diagnostics...
2026-06-05T11:52:20.347108407Z Gathering data for ns/openshift-network-node-identity...
2026-06-05T11:52:20.693471080Z Gathering data for ns/openshift-network-operator...
2026-06-05T11:52:21.032617717Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-05T11:52:21.264720362Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-05T11:52:21.700586789Z Gathering data for ns/openshift-apiserver-operator...
2026-06-05T11:52:21.973680614Z Gathering data for ns/openshift-apiserver...
2026-06-05T11:52:22.367569935Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-05T11:52:22.642672042Z Gathering data for ns/openshift-controller-manager...
2026-06-05T11:52:22.884750166Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-05T11:52:23.503301595Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-05T11:52:23.752197895Z Gathering data for ns/openshift-service-ca-operator...
2026-06-05T11:52:24.036343668Z Gathering data for ns/openshift-service-ca...
2026-06-05T11:52:24.363312708Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-05T11:52:24.663621204Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-05T11:52:24.848742648Z Collecting kubelet logs for node ip-10-0-140-113.ec2.internal
2026-06-05T11:52:24.850655954Z Collecting kubelet logs for node ip-10-0-129-98.ec2.internal
2026-06-05T11:52:24.852876157Z Collecting kubelet logs for node ip-10-0-135-216.ec2.internal
2026-06-05T11:52:25.174134653Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-hdqwv namespace
2026-06-05T11:52:25.179079543Z INFO: Node performance data collection complete.
2026-06-05T11:52:25.275703869Z Wrote inspect data to must-gather.
2026-06-05T11:52:25.275745557Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T11:52:25.275745557Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-05T11:52:25.411843580Z error: the server doesn't have a resource type "clusters"
2026-06-05T11:52:25.548224185Z Caches written to disk