2026-06-05T15:31:22.976494666Z [disk usage checker] Started
2026-06-05T15:31:22.979784313Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-06-05T15:31:23.182380439Z Gathering data for ns/openshift-cluster-version...
2026-06-05T15:31:23.293988555Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T15:31:23.384036124Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T15:31:23.460789441Z Gathering data for ns/default...
2026-06-05T15:31:23.769319539Z Gathering data for ns/openshift...
2026-06-05T15:31:24.170670904Z Gathering data for ns/kube-system...
2026-06-05T15:31:24.808472459Z Gathering data for ns/openshift-etcd...
2026-06-05T15:31:25.102599575Z Wrote inspect data to must-gather.
2026-06-05T15:31:25.102624285Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T15:31:25.102624285Z     namespaces "assisted-installer" not found
2026-06-05T15:31:25.441879138Z Waiting on subprocesses to finish execution.
2026-06-05T15:31:25.448449571Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-05T15:31:25.454476824Z Executing Istio gather script
2026-06-05T15:31:25.454679676Z INFO: Gathering HAProxy config files
2026-06-05T15:31:25.455679623Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-05T15:31:25.463561227Z INFO: Waiting for node performance related collection to complete ...
2026-06-05T15:31:25.468305939Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-05T15:31:25.492874846Z INFO: Collecting host service logs for crio
2026-06-05T15:31:25.493251025Z INFO: Collecting host service logs for kubelet
2026-06-05T15:31:25.493479498Z INFO: Collecting host service logs for rpm-ostreed
2026-06-05T15:31:25.493800089Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-05T15:31:25.494126954Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-05T15:31:25.494423952Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-05T15:31:25.494764753Z INFO: Collecting host service logs for NetworkManager
2026-06-05T15:31:25.495100987Z INFO: Collecting host service logs for openvswitch
2026-06-05T15:31:25.495449175Z INFO: Collecting host service logs for ovs-configuration
2026-06-05T15:31:25.499139827Z INFO: Collecting host service logs for ovsdb-server
2026-06-05T15:31:25.499691812Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-05T15:31:25.500346656Z INFO: Waiting for worker host service log collection to complete ...
2026-06-05T15:31:26.141529075Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-05T15:31:26.217159764Z No resources found in openshift-etcd namespace.
2026-06-05T15:31:26.250877370Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-05T15:31:26.250959479Z INFO: on-disk MachineConfig config collection complete.
2026-06-05T15:31:26.531939828Z INFO: "metallb-operator" not detected. Skipping.
2026-06-05T15:31:26.782508728Z ERROR: No running kube-apiserver pods found
2026-06-05T15:31:26.828626281Z INFO: Collecting Insights Archives from insights-operator-896549976-lhhgf
2026-06-05T15:31:26.828981145Z insights-runtime-extractor-29v9f
2026-06-05T15:31:26.829010838Z insights-runtime-extractor-c7k2m
2026-06-05T15:31:26.829074200Z insights-runtime-extractor-l4hjx
2026-06-05T15:31:26.858456103Z error: the server doesn't have a resource type "performanceprofile"
2026-06-05T15:31:26.882425918Z INFO: Worker host service log collection to complete.
2026-06-05T15:31:26.902578168Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-05T15:31:26.919550228Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-05T15:31:26.919626526Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-05T15:31:26.958346952Z Wrote inspect data to must-gather.
2026-06-05T15:31:27.037872424Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-05T15:31:27.054746965Z No resources found
2026-06-05T15:31:27.237944942Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-05T15:31:27.355868707Z Wrote inspect data to must-gather.
2026-06-05T15:31:27.483251711Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-05T15:31:27.483251711Z See 'oc rsync -h' for help and examples
2026-06-05T15:31:27.538421487Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-05T15:31:27.628946348Z Wrote inspect data to must-gather.
2026-06-05T15:31:27.746360675Z Wrote inspect data to must-gather.
2026-06-05T15:31:27.755450826Z Gathering data for ns/openshift-monitoring...
2026-06-05T15:31:27.761120031Z INFO: HAProxy config collection complete.
2026-06-05T15:31:27.769717587Z No resources found
2026-06-05T15:31:27.777840137Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T15:31:27.957869219Z Wrote inspect data to must-gather.
2026-06-05T15:31:28.011076297Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-06-05T15:31:28.012209891Z INFO: 'previous-logs' folder not found on ip-10-0-130-82.ec2.internal, skipping...
2026-06-05T15:31:28.023896819Z Wrote inspect data to must-gather.
2026-06-05T15:31:28.049939174Z error: the server doesn't have a resource type "machineconfigs"
2026-06-05T15:31:28.098609134Z Wrote inspect data to must-gather.
2026-06-05T15:31:28.231021499Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-05T15:31:28.264208326Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-05T15:31:28.306935149Z INFO: Getting rules from prometheus-k8s-0
2026-06-05T15:31:28.319741716Z Wrote inspect data to must-gather.
2026-06-05T15:31:28.320735538Z INFO: 'previous-logs' folder not found on ip-10-0-140-49.ec2.internal, skipping...
2026-06-05T15:31:28.506191744Z Wrote inspect data to must-gather.
2026-06-05T15:31:28.582233961Z Wrote inspect data to must-gather.
2026-06-05T15:31:28.582304635Z INFO: 'previous-logs' folder not found on ip-10-0-141-85.ec2.internal, skipping...
2026-06-05T15:31:28.582420820Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-05T15:31:28.582489621Z INFO: Machine Config Daemon termination log collection complete.
2026-06-05T15:31:28.672566769Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-05T15:31:28.713006431Z INFO: Getting status/config from prometheus-k8s-0
2026-06-05T15:31:28.827942250Z Wrote inspect data to must-gather.
2026-06-05T15:31:28.884784980Z Wrote inspect data to must-gather.
2026-06-05T15:31:29.106788826Z INFO: INTERCONNECT MODE
2026-06-05T15:31:29.106882336Z INFO: Gathering ovn-kubernetes DBs
2026-06-05T15:31:29.117586100Z Wrote inspect data to must-gather.
2026-06-05T15:31:29.126801495Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-05T15:31:29.196727004Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-05T15:31:29.286332908Z INFO: Gathering OVN_Northbound from ovnkube-node-2hmjd...
2026-06-05T15:31:29.286680607Z INFO: Gathering OVN_Northbound from ovnkube-node-2hqs2...
2026-06-05T15:31:29.286964517Z INFO: Gathering OVN_Northbound from ovnkube-node-wcsrx...
2026-06-05T15:31:29.287375804Z INFO: Gathering OVN_Southbound from ovnkube-node-2hmjd...
2026-06-05T15:31:29.287679214Z INFO: Gathering OVN_Southbound from ovnkube-node-2hqs2...
2026-06-05T15:31:29.287948517Z INFO: Gathering OVN_Southbound from ovnkube-node-wcsrx...
2026-06-05T15:31:29.306724573Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T15:31:29.376445172Z Wrote inspect data to must-gather.
2026-06-05T15:31:29.394873330Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-05T15:31:29.487288782Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:29.568641005Z tar: Removing leading `/' from member names
2026-06-05T15:31:29.577724375Z tar: Removing leading `/' from member names
2026-06-05T15:31:29.608961762Z tar: Removing leading `/' from member names
2026-06-05T15:31:29.613730066Z tar: Removing leading `/' from member names
2026-06-05T15:31:29.617288992Z tar: Removing leading `/' from member names
2026-06-05T15:31:29.627236434Z tar: Removing leading `/' from member names
2026-06-05T15:31:29.697778038Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-05T15:31:29.726260346Z Wrote inspect data to must-gather.
2026-06-05T15:31:29.745962960Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:29.749126868Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-05T15:31:29.956138529Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-05T15:31:29.981206799Z W0605 15:31:29.981157    1407 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-05T15:31:29.985819675Z Wrote inspect data to must-gather.
2026-06-05T15:31:29.994879663Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:30.135392109Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T15:31:30.139435019Z INFO: Waiting for network log collection to complete ...
2026-06-05T15:31:30.139489428Z INFO: Waiting for ovnk database copies to complete ...
2026-06-05T15:31:30.139979988Z INFO: Copying ovnk databases complete.
2026-06-05T15:31:30.141746978Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-06-05T15:31:30.145412889Z ovnk_database_store/
2026-06-05T15:31:30.145412889Z ovnk_database_store/ovnkube-node-2hmjd_sbdb
2026-06-05T15:31:30.166139746Z ovnk_database_store/ovnkube-node-2hmjd_nbdb
2026-06-05T15:31:30.174432995Z ovnk_database_store/ovnkube-node-wcsrx_nbdb
2026-06-05T15:31:30.183216970Z ovnk_database_store/ovnkube-node-2hqs2_nbdb
2026-06-05T15:31:30.191807023Z ovnk_database_store/ovnkube-node-2hqs2_sbdb
2026-06-05T15:31:30.212243296Z ovnk_database_store/ovnkube-node-wcsrx_sbdb
2026-06-05T15:31:30.231175212Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:30.234216705Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-05T15:31:30.234390981Z INFO: Network log collection complete.
2026-06-05T15:31:30.316814688Z Wrote inspect data to must-gather.
2026-06-05T15:31:30.323810458Z Gathering data for ns/openshift-network-console...
2026-06-05T15:31:30.349583796Z INFO: Getting status from alertmanager-main-0
2026-06-05T15:31:30.447659406Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:30.625392118Z Wrote inspect data to must-gather.
2026-06-05T15:31:30.648075329Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:30.651033449Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-05T15:31:30.795972363Z Gathering data for ns/openshift-console-operator...
2026-06-05T15:31:30.884645152Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:30.897823790Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-05T15:31:30.900421987Z INFO: Fallback to identify the container image from release info
2026-06-05T15:31:31.067160956Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:31.069830193Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-05T15:31:31.212810374Z Gathering data for ns/openshift-console...
2026-06-05T15:31:31.230736771Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:31.409340885Z Wrote inspect data to /must-gather/istio.
2026-06-05T15:31:31.527584935Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-05T15:31:31.662652318Z daemonset.apps/perf-node-gather-daemonset created
2026-06-05T15:31:31.768187452Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-05T15:31:31.797542377Z Waiting for performance profile collector pods to become ready: 1
2026-06-05T15:31:32.113128686Z Done executing Istio gather script
2026-06-05T15:31:32.308646070Z Gathering data for ns/openshift-dns-operator...
2026-06-05T15:31:32.581879040Z Gathering data for ns/openshift-dns...
2026-06-05T15:31:32.914051156Z Waiting for performance profile collector pods to become ready: 2
2026-06-05T15:31:33.017830859Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-06-05T15:31:33.147554229Z Gathering data for ns/openshift-image-registry...
2026-06-05T15:31:33.582957451Z Gathering data for ns/openshift-ingress-operator...
2026-06-05T15:31:33.942536045Z Gathering data for ns/openshift-ingress...
2026-06-05T15:31:34.025789831Z Waiting for performance profile collector pods to become ready: 3
2026-06-05T15:31:34.267737105Z Gathering data for ns/openshift-ingress-canary...
2026-06-05T15:31:34.632059474Z Gathering data for ns/openshift-insights...
2026-06-05T15:31:35.140951086Z Waiting for performance profile collector pods to become ready: 4
2026-06-05T15:31:36.251660995Z Waiting for performance profile collector pods to become ready: 5
2026-06-05T15:31:36.428541316Z Gathering data for ns/openshift-jobset-operator...
2026-06-05T15:31:37.174216883Z Gathering data for ns/openshift-config...
2026-06-05T15:31:37.348010905Z Waiting for performance profile collector pods to become ready: 6
2026-06-05T15:31:37.513624460Z Gathering data for ns/openshift-config-managed...
2026-06-05T15:31:37.970259284Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-05T15:31:38.024325288Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-06-05T15:31:38.264779267Z Gathering data for ns/openshift-kube-apiserver...
2026-06-05T15:31:38.457067031Z Waiting for performance profile collector pods to become ready: 7
2026-06-05T15:31:38.526188501Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-05T15:31:38.832780643Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-05T15:31:39.154337547Z Gathering data for ns/openshift-kube-scheduler...
2026-06-05T15:31:39.453537726Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-05T15:31:39.575638034Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-05T15:31:39.757689702Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-05T15:31:39.831000983Z Collecting performance related data for node ip-10-0-130-82.ec2.internal
2026-06-05T15:31:39.840120585Z Collecting performance related data for node ip-10-0-140-49.ec2.internal
2026-06-05T15:31:39.848887522Z Collecting performance related data for node ip-10-0-141-85.ec2.internal
2026-06-05T15:31:40.095965134Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-05T15:31:40.445619318Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-05T15:31:41.052009079Z Gathering data for ns/openshift-multus...
2026-06-05T15:31:42.507795426Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-05T15:31:43.029141399Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-06-05T15:31:43.802114639Z Gathering data for ns/openshift-host-network...
2026-06-05T15:31:44.162469817Z Gathering data for ns/openshift-network-diagnostics...
2026-06-05T15:31:44.704405832Z Gathering data for ns/openshift-network-node-identity...
2026-06-05T15:31:45.112085342Z Gathering data for ns/openshift-network-operator...
2026-06-05T15:31:45.489325058Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-05T15:31:45.742587860Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-05T15:31:46.207854109Z Gathering data for ns/openshift-apiserver-operator...
2026-06-05T15:31:46.567072287Z Gathering data for ns/openshift-apiserver...
2026-06-05T15:31:46.992628924Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-05T15:31:47.312804370Z Gathering data for ns/openshift-controller-manager...
2026-06-05T15:31:47.666758757Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-05T15:31:47.836938257Z Collecting kubelet logs for node ip-10-0-130-82.ec2.internal
2026-06-05T15:31:47.838934861Z Collecting kubelet logs for node ip-10-0-140-49.ec2.internal
2026-06-05T15:31:47.841001909Z Collecting kubelet logs for node ip-10-0-141-85.ec2.internal
2026-06-05T15:31:48.035459043Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-06-05T15:31:48.159792649Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-hcmzz namespace
2026-06-05T15:31:48.164100932Z INFO: Node performance data collection complete.
2026-06-05T15:31:48.427261422Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-05T15:31:48.703139689Z Gathering data for ns/openshift-service-ca-operator...
2026-06-05T15:31:48.997589598Z Gathering data for ns/openshift-service-ca...
2026-06-05T15:31:49.339130713Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-05T15:31:50.166773400Z Wrote inspect data to must-gather.
2026-06-05T15:31:50.166819997Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T15:31:50.166819997Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-05T15:31:50.287756886Z error: the server doesn't have a resource type "clusters"
2026-06-05T15:31:50.418055197Z Caches written to disk