2026-06-12T19:28:18.399085431Z [disk usage checker] Started
2026-06-12T19:28:18.402767343Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T19:28:18.599048657Z Gathering data for ns/openshift-cluster-version...
2026-06-12T19:28:18.736338141Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-12T19:28:18.825054571Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-12T19:28:18.908504593Z Gathering data for ns/default...
2026-06-12T19:28:19.184976645Z Gathering data for ns/openshift...
2026-06-12T19:28:19.608956771Z Gathering data for ns/kube-system...
2026-06-12T19:28:20.332585091Z Gathering data for ns/openshift-etcd...
2026-06-12T19:28:20.628065221Z Wrote inspect data to must-gather.
2026-06-12T19:28:20.628157865Z error: inspection completed with the errors occurred while gathering data:
2026-06-12T19:28:20.628157865Z     namespaces "assisted-installer" not found
2026-06-12T19:28:20.972421543Z Waiting on subprocesses to finish execution.
2026-06-12T19:28:20.982762361Z INFO: Collecting host service logs for crio
2026-06-12T19:28:20.983337036Z INFO: Collecting host service logs for kubelet
2026-06-12T19:28:20.983711773Z INFO: Collecting host service logs for rpm-ostreed
2026-06-12T19:28:20.984076397Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-12T19:28:20.984428505Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-12T19:28:20.984780646Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-12T19:28:20.985158627Z INFO: Collecting host service logs for NetworkManager
2026-06-12T19:28:20.985513346Z INFO: Collecting host service logs for openvswitch
2026-06-12T19:28:20.985924629Z INFO: Collecting host service logs for ovs-configuration
2026-06-12T19:28:20.985980228Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-12T19:28:20.986305719Z INFO: Collecting host service logs for ovsdb-server
2026-06-12T19:28:20.986655562Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-12T19:28:20.987063774Z INFO: Waiting for worker host service log collection to complete ...
2026-06-12T19:28:20.987762652Z Executing Istio gather script
2026-06-12T19:28:20.989144423Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-12T19:28:21.005062488Z INFO: Gathering HAProxy config files
2026-06-12T19:28:21.038062526Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-12T19:28:21.279809759Z INFO: Waiting for node performance related collection to complete ...
2026-06-12T19:28:21.570740795Z No resources found in openshift-etcd namespace.
2026-06-12T19:28:21.740048842Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-12T19:28:21.929507164Z INFO: Collecting Insights Archives from insights-operator-5bbd86d6bd-sz85v
2026-06-12T19:28:21.929569596Z insights-runtime-extractor-42w8x
2026-06-12T19:28:21.929598749Z insights-runtime-extractor-kzvt6
2026-06-12T19:28:21.929628206Z insights-runtime-extractor-q267c
2026-06-12T19:28:22.011256334Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-12T19:28:22.070711608Z No resources found
2026-06-12T19:28:22.285583781Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-12T19:28:22.325779172Z INFO: Worker host service log collection to complete.
2026-06-12T19:28:22.346561705Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-12T19:28:22.346684794Z INFO: on-disk MachineConfig config collection complete.
2026-06-12T19:28:22.395489102Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-12T19:28:22.469543597Z INFO: "metallb-operator" not detected. Skipping.
2026-06-12T19:28:22.492553103Z Wrote inspect data to must-gather.
2026-06-12T19:28:22.529085588Z Wrote inspect data to must-gather.
2026-06-12T19:28:22.540703468Z error: the server doesn't have a resource type "performanceprofile"
2026-06-12T19:28:22.811779611Z ERROR: No running kube-apiserver pods found
2026-06-12T19:28:23.022106883Z Wrote inspect data to must-gather.
2026-06-12T19:28:23.094603165Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-12T19:28:23.094603165Z See 'oc rsync -h' for help and examples
2026-06-12T19:28:23.098476709Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-12T19:28:23.103766052Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-12T19:28:23.129767971Z INFO: 'previous-logs' folder not found on ip-10-0-131-207.ec2.internal, skipping...
2026-06-12T19:28:23.141760937Z Wrote inspect data to must-gather.
2026-06-12T19:28:23.189976987Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-12T19:28:23.230635142Z Wrote inspect data to must-gather.
2026-06-12T19:28:23.241652327Z Wrote inspect data to must-gather.
2026-06-12T19:28:23.255726830Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-12T19:28:23.293509409Z No resources found
2026-06-12T19:28:23.422958810Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T19:28:23.471991835Z Wrote inspect data to must-gather.
2026-06-12T19:28:23.503343303Z Wrote inspect data to must-gather.
2026-06-12T19:28:23.519120350Z INFO: 'previous-logs' folder not found on ip-10-0-139-222.ec2.internal, skipping...
2026-06-12T19:28:23.556004107Z Gathering data for ns/openshift-monitoring...
2026-06-12T19:28:23.582028178Z tar: Removing leading `/' from member names
2026-06-12T19:28:23.585940474Z INFO: HAProxy config collection complete.
2026-06-12T19:28:23.690315982Z error: the server doesn't have a resource type "machineconfigs"
2026-06-12T19:28:23.769526166Z INFO: Getting rules from prometheus-k8s-0
2026-06-12T19:28:23.803135249Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-12T19:28:23.813398346Z Wrote inspect data to must-gather.
2026-06-12T19:28:23.844613133Z INFO: 'previous-logs' folder not found on ip-10-0-141-100.ec2.internal, skipping...
2026-06-12T19:28:23.847011542Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-12T19:28:23.847011542Z INFO: Machine Config Daemon termination log collection complete.
2026-06-12T19:28:23.892120463Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-12T19:28:24.053929571Z Wrote inspect data to must-gather.
2026-06-12T19:28:24.099808556Z Wrote inspect data to must-gather.
2026-06-12T19:28:24.184019991Z INFO: Getting status/config from prometheus-k8s-0
2026-06-12T19:28:24.249105552Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-12T19:28:24.341578422Z INFO: INTERCONNECT MODE
2026-06-12T19:28:24.341617460Z INFO: Gathering ovn-kubernetes DBs
2026-06-12T19:28:24.460251744Z Wrote inspect data to must-gather.
2026-06-12T19:28:24.493466993Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-12T19:28:24.497623626Z INFO: Gathering OVN_Northbound from ovnkube-node-5xf9p...
2026-06-12T19:28:24.497888817Z INFO: Gathering OVN_Northbound from ovnkube-node-nxlr8...
2026-06-12T19:28:24.498223790Z INFO: Gathering OVN_Northbound from ovnkube-node-qrxjj...
2026-06-12T19:28:24.498524847Z INFO: Gathering OVN_Southbound from ovnkube-node-5xf9p...
2026-06-12T19:28:24.498790417Z INFO: Gathering OVN_Southbound from ovnkube-node-nxlr8...
2026-06-12T19:28:24.499055259Z INFO: Gathering OVN_Southbound from ovnkube-node-qrxjj...
2026-06-12T19:28:24.702204043Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-12T19:28:24.805985638Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:24.840677075Z tar: Removing leading `/' from member names
2026-06-12T19:28:24.840713550Z tar: Removing leading `/' from member names
2026-06-12T19:28:24.845112804Z tar: Removing leading `/' from member names
2026-06-12T19:28:24.855571635Z Wrote inspect data to must-gather.
2026-06-12T19:28:24.858950287Z tar: Removing leading `/' from member names
2026-06-12T19:28:24.880505641Z tar: Removing leading `/' from member names
2026-06-12T19:28:24.885303074Z tar: Removing leading `/' from member names
2026-06-12T19:28:24.999255247Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-12T19:28:25.039527332Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:25.042526128Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-12T19:28:25.085597849Z Wrote inspect data to must-gather.
2026-06-12T19:28:25.249435421Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-12T19:28:25.267046159Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:25.329159129Z Wrote inspect data to must-gather.
2026-06-12T19:28:25.349425960Z INFO: Waiting for network log collection to complete ...
2026-06-12T19:28:25.349469910Z INFO: Waiting for ovnk database copies to complete ...
2026-06-12T19:28:25.350019627Z INFO: Copying ovnk databases complete.
2026-06-12T19:28:25.351624599Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-06-12T19:28:25.354214214Z ovnk_database_store/
2026-06-12T19:28:25.354214214Z ovnk_database_store/ovnkube-node-qrxjj_sbdb
2026-06-12T19:28:25.369709386Z ovnk_database_store/ovnkube-node-nxlr8_sbdb
2026-06-12T19:28:25.385932494Z ovnk_database_store/ovnkube-node-nxlr8_nbdb
2026-06-12T19:28:25.392645314Z ovnk_database_store/ovnkube-node-qrxjj_nbdb
2026-06-12T19:28:25.399354188Z ovnk_database_store/ovnkube-node-5xf9p_nbdb
2026-06-12T19:28:25.405684494Z ovnk_database_store/ovnkube-node-5xf9p_sbdb
2026-06-12T19:28:25.422314922Z INFO: Network log collection complete.
2026-06-12T19:28:25.424385764Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-12T19:28:25.487934367Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:25.490316242Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-12T19:28:25.506049548Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-12T19:28:25.540066952Z Wrote inspect data to must-gather.
2026-06-12T19:28:25.689966395Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:25.738519907Z Wrote inspect data to must-gather.
2026-06-12T19:28:25.895415842Z INFO: Getting status from alertmanager-main-0
2026-06-12T19:28:25.901457520Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:25.904592349Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-12T19:28:25.955786315Z W0612 19:28:25.955735    1538 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-12T19:28:25.963056180Z Wrote inspect data to must-gather.
2026-06-12T19:28:26.008279672Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-12T19:28:26.138536979Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:26.211319614Z Gathering data for ns/openshift-network-console...
2026-06-12T19:28:26.228146919Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-12T19:28:26.230775883Z INFO: Fallback to identify the container image from release info
2026-06-12T19:28:26.348178994Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:26.350829056Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-12T19:28:26.506646944Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:26.636914307Z Gathering data for ns/openshift-console-operator...
2026-06-12T19:28:26.708448904Z Wrote inspect data to /must-gather/istio.
2026-06-12T19:28:26.746479843Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d66a6b2ec52dede5d83ce04f6abfc519df1fd7dc36248741e03ecb9bf8d1762
2026-06-12T19:28:26.861671616Z daemonset.apps/perf-node-gather-daemonset created
2026-06-12T19:28:27.001776153Z Waiting for performance profile collector pods to become ready: 1
2026-06-12T19:28:27.060503986Z Gathering data for ns/openshift-console...
2026-06-12T19:28:27.388913989Z Done executing Istio gather script
2026-06-12T19:28:27.588125215Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-12T19:28:28.119401670Z Gathering data for ns/openshift-dns-operator...
2026-06-12T19:28:28.127340022Z Waiting for performance profile collector pods to become ready: 2
2026-06-12T19:28:28.411208456Z Gathering data for ns/openshift-dns...
2026-06-12T19:28:28.445721197Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T19:28:29.059000919Z Gathering data for ns/openshift-image-registry...
2026-06-12T19:28:29.242477663Z Waiting for performance profile collector pods to become ready: 3
2026-06-12T19:28:29.539467230Z Gathering data for ns/openshift-ingress-operator...
2026-06-12T19:28:29.864906525Z Gathering data for ns/openshift-ingress...
2026-06-12T19:28:30.192219768Z Gathering data for ns/openshift-ingress-canary...
2026-06-12T19:28:30.365867510Z Waiting for performance profile collector pods to become ready: 4
2026-06-12T19:28:30.597364144Z Gathering data for ns/openshift-insights...
2026-06-12T19:28:31.476079493Z Waiting for performance profile collector pods to become ready: 5
2026-06-12T19:28:32.442206444Z Gathering data for ns/openshift-config...
2026-06-12T19:28:32.587464012Z Waiting for performance profile collector pods to become ready: 6
2026-06-12T19:28:32.690607697Z Gathering data for ns/openshift-config-managed...
2026-06-12T19:28:33.104085435Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-12T19:28:33.377505238Z Gathering data for ns/openshift-kube-apiserver...
2026-06-12T19:28:33.450302692Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T19:28:33.623348601Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-12T19:28:33.714872699Z Waiting for performance profile collector pods to become ready: 7
2026-06-12T19:28:33.898716073Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-12T19:28:34.183213745Z Gathering data for ns/openshift-kube-scheduler...
2026-06-12T19:28:34.472100555Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-12T19:28:34.740699636Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-12T19:28:34.829770564Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-12T19:28:35.086316397Z Collecting performance related data for node ip-10-0-131-207.ec2.internal
2026-06-12T19:28:35.086879169Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-12T19:28:35.093163539Z Collecting performance related data for node ip-10-0-139-222.ec2.internal
2026-06-12T19:28:35.101200489Z Collecting performance related data for node ip-10-0-141-100.ec2.internal
2026-06-12T19:28:35.453968889Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-12T19:28:36.042480931Z Gathering data for ns/openshift-multus...
2026-06-12T19:28:37.591413793Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-12T19:28:38.455315083Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T19:28:38.912220461Z Gathering data for ns/openshift-host-network...
2026-06-12T19:28:39.219004988Z Gathering data for ns/openshift-network-diagnostics...
2026-06-12T19:28:39.745813560Z Gathering data for ns/openshift-network-node-identity...
2026-06-12T19:28:40.156397399Z Gathering data for ns/openshift-network-operator...
2026-06-12T19:28:40.598019840Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-12T19:28:40.869834639Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-12T19:28:41.331265524Z Gathering data for ns/openshift-apiserver-operator...
2026-06-12T19:28:41.628261393Z Gathering data for ns/openshift-apiserver...
2026-06-12T19:28:42.009199552Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-12T19:28:42.317477467Z Gathering data for ns/openshift-controller-manager...
2026-06-12T19:28:42.577976006Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-12T19:28:43.237110192Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-12T19:28:43.461144142Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T19:28:43.464460381Z Collecting kubelet logs for node ip-10-0-131-207.ec2.internal
2026-06-12T19:28:43.466164141Z Collecting kubelet logs for node ip-10-0-139-222.ec2.internal
2026-06-12T19:28:43.468644620Z Collecting kubelet logs for node ip-10-0-141-100.ec2.internal
2026-06-12T19:28:43.585376176Z Gathering data for ns/openshift-service-ca-operator...
2026-06-12T19:28:43.818647698Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-dnzrd namespace
2026-06-12T19:28:43.823938253Z INFO: Node performance data collection complete.
2026-06-12T19:28:43.957607462Z Gathering data for ns/openshift-service-ca...
2026-06-12T19:28:44.333759837Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-12T19:28:45.228396690Z Wrote inspect data to must-gather.
2026-06-12T19:28:45.228444622Z error: inspection completed with the errors occurred while gathering data:
2026-06-12T19:28:45.228444622Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-12T19:28:45.348480798Z error: the server doesn't have a resource type "clusters"
2026-06-12T19:28:45.470411764Z Caches written to disk