2026-07-09T07:56:32.162634356Z [disk usage checker] Started
2026-07-09T07:56:32.166057612Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-09T07:56:32.362670968Z Gathering data for ns/openshift-cluster-version...
2026-07-09T07:56:32.500727616Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-09T07:56:32.596443218Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-09T07:56:32.681881918Z Gathering data for ns/default...
2026-07-09T07:56:32.985202429Z Gathering data for ns/openshift...
2026-07-09T07:56:33.427136398Z Gathering data for ns/kube-system...
2026-07-09T07:56:34.353485729Z Gathering data for ns/openshift-etcd...
2026-07-09T07:56:34.688349596Z Waiting on subprocesses to finish execution.
2026-07-09T07:56:34.693964247Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-09T07:56:34.696379015Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-09T07:56:34.710228949Z Wrote inspect data to must-gather.
2026-07-09T07:56:34.710314014Z error: inspection completed with the errors occurred while gathering data:
2026-07-09T07:56:34.710314014Z     namespaces "assisted-installer" not found
2026-07-09T07:56:34.711938793Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-09T07:56:34.717913236Z Executing Istio gather script
2026-07-09T07:56:34.728156636Z INFO: Collecting host service logs for crio
2026-07-09T07:56:34.728519746Z INFO: Collecting host service logs for kubelet
2026-07-09T07:56:34.728840794Z INFO: Collecting host service logs for rpm-ostreed
2026-07-09T07:56:34.729153170Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-09T07:56:34.729469248Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-09T07:56:34.729777113Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-09T07:56:34.730163344Z INFO: Collecting host service logs for NetworkManager
2026-07-09T07:56:34.733052328Z INFO: Collecting host service logs for openvswitch
2026-07-09T07:56:34.733438267Z INFO: Collecting host service logs for ovs-configuration
2026-07-09T07:56:34.734274022Z INFO: Collecting host service logs for ovsdb-server
2026-07-09T07:56:34.736558553Z INFO: Gathering HAProxy config files
2026-07-09T07:56:34.745605282Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-09T07:56:34.746021059Z INFO: Waiting for worker host service log collection to complete ...
2026-07-09T07:56:35.134749621Z INFO: Waiting for node performance related collection to complete ...
2026-07-09T07:56:35.249545085Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-09T07:56:35.348440132Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-8h4qb
2026-07-09T07:56:35.348493889Z insights-runtime-extractor-8sfws
2026-07-09T07:56:35.348515057Z insights-runtime-extractor-p84dc
2026-07-09T07:56:35.348534976Z insights-runtime-extractor-x8b2d
2026-07-09T07:56:35.624237704Z No resources found
2026-07-09T07:56:35.679459286Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-09T07:56:35.840421189Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-09T07:56:35.840535745Z INFO: on-disk MachineConfig config collection complete.
2026-07-09T07:56:35.961769474Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-09T07:56:36.032034289Z No resources found in openshift-etcd namespace.
2026-07-09T07:56:36.109915018Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-09T07:56:36.210610826Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-09T07:56:36.210610826Z See 'oc rsync -h' for help and examples
2026-07-09T07:56:36.258501937Z error: the server doesn't have a resource type "performanceprofile"
2026-07-09T07:56:36.457596256Z INFO: "metallb-operator" not detected. Skipping.
2026-07-09T07:56:36.500785257Z Wrote inspect data to must-gather.
2026-07-09T07:56:36.593916622Z No resources found
2026-07-09T07:56:36.603879991Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-09T07:56:36.624856743Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-09T07:56:36.659143348Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-09T07:56:36.739246042Z INFO: Worker host service log collection to complete.
2026-07-09T07:56:36.765042095Z Wrote inspect data to must-gather.
2026-07-09T07:56:36.854763820Z Wrote inspect data to must-gather.
2026-07-09T07:56:36.869202646Z Wrote inspect data to must-gather.
2026-07-09T07:56:36.888411213Z INFO: 'previous-logs' folder not found on ip-10-0-136-139.ec2.internal, skipping...
2026-07-09T07:56:36.913397779Z ERROR: No running kube-apiserver pods found
2026-07-09T07:56:36.971219827Z Wrote inspect data to must-gather.
2026-07-09T07:56:36.988571304Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-09T07:56:37.173508487Z Wrote inspect data to must-gather.
2026-07-09T07:56:37.184620973Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-09T07:56:37.210446193Z Wrote inspect data to must-gather.
2026-07-09T07:56:37.214940205Z tar: Removing leading `/' from member names
2026-07-09T07:56:37.221184418Z INFO: HAProxy config collection complete.
2026-07-09T07:56:37.270966361Z Gathering data for ns/openshift-monitoring...
2026-07-09T07:56:37.294676571Z INFO: 'previous-logs' folder not found on ip-10-0-140-202.ec2.internal, skipping...
2026-07-09T07:56:37.346686622Z Wrote inspect data to must-gather.
2026-07-09T07:56:37.431231463Z error: the server doesn't have a resource type "machineconfigs"
2026-07-09T07:56:37.494916500Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-09T07:56:37.514454734Z INFO: Getting rules from prometheus-k8s-0
2026-07-09T07:56:37.588202824Z INFO: 'previous-logs' folder not found on ip-10-0-143-198.ec2.internal, skipping...
2026-07-09T07:56:37.588240660Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-09T07:56:37.588267244Z INFO: Machine Config Daemon termination log collection complete.
2026-07-09T07:56:37.624866743Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-09T07:56:37.635166960Z Wrote inspect data to must-gather.
2026-07-09T07:56:37.849860049Z Wrote inspect data to must-gather.
2026-07-09T07:56:37.878334916Z Wrote inspect data to must-gather.
2026-07-09T07:56:37.967945540Z INFO: Getting status/config from prometheus-k8s-0
2026-07-09T07:56:38.003084495Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-09T07:56:38.143636074Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-09T07:56:38.179681676Z INFO: INTERCONNECT MODE
2026-07-09T07:56:38.179681676Z INFO: Gathering ovn-kubernetes DBs
2026-07-09T07:56:38.207468949Z Wrote inspect data to must-gather.
2026-07-09T07:56:38.327084809Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:38.348097591Z INFO: Gathering OVN_Northbound from ovnkube-node-7zn76...
2026-07-09T07:56:38.348403849Z INFO: Gathering OVN_Northbound from ovnkube-node-dc9wg...
2026-07-09T07:56:38.348745191Z INFO: Gathering OVN_Northbound from ovnkube-node-sr6xj...
2026-07-09T07:56:38.349126662Z INFO: Gathering OVN_Southbound from ovnkube-node-7zn76...
2026-07-09T07:56:38.349434692Z INFO: Gathering OVN_Southbound from ovnkube-node-dc9wg...
2026-07-09T07:56:38.349734169Z INFO: Gathering OVN_Southbound from ovnkube-node-sr6xj...
2026-07-09T07:56:38.488778281Z Wrote inspect data to must-gather.
2026-07-09T07:56:38.508263643Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-09T07:56:38.667080922Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:38.670403606Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-09T07:56:38.708292138Z tar: Removing leading `/' from member names
2026-07-09T07:56:38.710688590Z tar: Removing leading `/' from member names
2026-07-09T07:56:38.714379216Z tar: Removing leading `/' from member names
2026-07-09T07:56:38.714704443Z tar: Removing leading `/' from member names
2026-07-09T07:56:38.737522160Z tar: Removing leading `/' from member names
2026-07-09T07:56:38.743691776Z tar: Removing leading `/' from member names
2026-07-09T07:56:38.813762308Z Wrote inspect data to must-gather.
2026-07-09T07:56:38.841896698Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-09T07:56:38.915301057Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:39.080422429Z Wrote inspect data to must-gather.
2026-07-09T07:56:39.164166224Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-09T07:56:39.181202729Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:39.183679234Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-09T07:56:39.198872666Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-09T07:56:39.210553904Z INFO: Waiting for network log collection to complete ...
2026-07-09T07:56:39.210600716Z INFO: Waiting for ovnk database copies to complete ...
2026-07-09T07:56:39.211076621Z INFO: Copying ovnk databases complete.
2026-07-09T07:56:39.212661560Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-09T07:56:39.215127367Z ovnk_database_store/
2026-07-09T07:56:39.215174091Z ovnk_database_store/ovnkube-node-sr6xj_nbdb
2026-07-09T07:56:39.223549594Z ovnk_database_store/ovnkube-node-7zn76_sbdb
2026-07-09T07:56:39.239668591Z ovnk_database_store/ovnkube-node-sr6xj_sbdb
2026-07-09T07:56:39.258496406Z ovnk_database_store/ovnkube-node-7zn76_nbdb
2026-07-09T07:56:39.263969179Z ovnk_database_store/ovnkube-node-dc9wg_sbdb
2026-07-09T07:56:39.281735818Z ovnk_database_store/ovnkube-node-dc9wg_nbdb
2026-07-09T07:56:39.292347358Z INFO: Network log collection complete.
2026-07-09T07:56:39.321552632Z Wrote inspect data to must-gather.
2026-07-09T07:56:39.383946724Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:39.458296804Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-09T07:56:39.510596226Z Wrote inspect data to must-gather.
2026-07-09T07:56:39.595753543Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:39.598020712Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-09T07:56:39.697167652Z W0709 07:56:39.697125    1556 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-09T07:56:39.702237502Z Wrote inspect data to must-gather.
2026-07-09T07:56:39.819111864Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:39.834004466Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-09T07:56:39.896702836Z INFO: Getting status from alertmanager-main-0
2026-07-09T07:56:39.988525801Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-09T07:56:39.991441242Z INFO: Fallback to identify the container image from release info
2026-07-09T07:56:40.026574536Z Gathering data for ns/openshift-network-console...
2026-07-09T07:56:40.037122765Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:40.039871800Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-09T07:56:40.207769391Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:40.395775283Z Wrote inspect data to /must-gather/istio.
2026-07-09T07:56:40.479020872Z Gathering data for ns/openshift-console-operator...
2026-07-09T07:56:40.636614391Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-09T07:56:40.776874151Z daemonset.apps/perf-node-gather-daemonset created
2026-07-09T07:56:40.921155684Z Gathering data for ns/openshift-console...
2026-07-09T07:56:40.933990025Z Waiting for performance profile collector pods to become ready: 1
2026-07-09T07:56:41.084292651Z Done executing Istio gather script
2026-07-09T07:56:41.406053163Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-09T07:56:41.886490281Z Gathering data for ns/openshift-dns-operator...
2026-07-09T07:56:42.042903479Z Waiting for performance profile collector pods to become ready: 2
2026-07-09T07:56:42.176422288Z Gathering data for ns/openshift-dns...
2026-07-09T07:56:42.189451476Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-09T07:56:42.733411017Z Gathering data for ns/openshift-image-registry...
2026-07-09T07:56:43.161922703Z Waiting for performance profile collector pods to become ready: 3
2026-07-09T07:56:43.180350404Z Gathering data for ns/openshift-ingress-operator...
2026-07-09T07:56:43.482550113Z Gathering data for ns/openshift-ingress...
2026-07-09T07:56:43.867503600Z Gathering data for ns/openshift-ingress-canary...
2026-07-09T07:56:44.271748878Z Gathering data for ns/openshift-insights...
2026-07-09T07:56:44.286614337Z Waiting for performance profile collector pods to become ready: 4
2026-07-09T07:56:45.391489044Z Waiting for performance profile collector pods to become ready: 5
2026-07-09T07:56:46.223439544Z Gathering data for ns/openshift-config...
2026-07-09T07:56:46.509312918Z Waiting for performance profile collector pods to become ready: 6
2026-07-09T07:56:46.586710239Z Gathering data for ns/openshift-config-managed...
2026-07-09T07:56:47.162864033Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-09T07:56:47.195488772Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-09T07:56:47.480661506Z Gathering data for ns/openshift-kube-apiserver...
2026-07-09T07:56:47.617446010Z Waiting for performance profile collector pods to become ready: 7
2026-07-09T07:56:47.768526215Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-09T07:56:48.047469903Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-09T07:56:48.328438756Z Gathering data for ns/openshift-kube-scheduler...
2026-07-09T07:56:48.580415860Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-09T07:56:48.722848105Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-09T07:56:48.907065603Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-09T07:56:48.971286419Z Collecting performance related data for node ip-10-0-136-139.ec2.internal
2026-07-09T07:56:48.981003488Z Collecting performance related data for node ip-10-0-140-202.ec2.internal
2026-07-09T07:56:48.992139667Z Collecting performance related data for node ip-10-0-143-198.ec2.internal
2026-07-09T07:56:49.442936043Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-09T07:56:49.978408011Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-09T07:56:50.562637460Z Gathering data for ns/openshift-multus...
2026-07-09T07:56:52.204163488Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-09T07:56:52.633013944Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-09T07:56:54.409120007Z Gathering data for ns/openshift-host-network...
2026-07-09T07:56:54.776528677Z Gathering data for ns/openshift-network-diagnostics...
2026-07-09T07:56:55.478855893Z Gathering data for ns/openshift-network-node-identity...
2026-07-09T07:56:55.954988163Z Gathering data for ns/openshift-network-operator...
2026-07-09T07:56:56.526828649Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-09T07:56:56.864024087Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-09T07:56:57.144612466Z Collecting kubelet logs for node ip-10-0-136-139.ec2.internal
2026-07-09T07:56:57.146407457Z Collecting kubelet logs for node ip-10-0-140-202.ec2.internal
2026-07-09T07:56:57.148284671Z Collecting kubelet logs for node ip-10-0-143-198.ec2.internal
2026-07-09T07:56:57.209672721Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-09T07:56:57.517114988Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-pcsc4 namespace
2026-07-09T07:56:57.527029122Z INFO: Node performance data collection complete.
2026-07-09T07:56:57.584453088Z Gathering data for ns/openshift-apiserver-operator...
2026-07-09T07:56:57.973554682Z Gathering data for ns/openshift-apiserver...
2026-07-09T07:56:58.418265155Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-09T07:56:58.807025305Z Gathering data for ns/openshift-controller-manager...
2026-07-09T07:56:59.076827550Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-09T07:56:59.772457389Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-09T07:57:00.091189760Z Gathering data for ns/openshift-service-ca-operator...
2026-07-09T07:57:00.522406912Z Gathering data for ns/openshift-service-ca...
2026-07-09T07:57:00.868935992Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-09T07:57:01.731454277Z Wrote inspect data to must-gather.
2026-07-09T07:57:01.731498387Z error: inspection completed with the errors occurred while gathering data:
2026-07-09T07:57:01.731498387Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-09T07:57:01.851545214Z error: the server doesn't have a resource type "clusters"
2026-07-09T07:57:02.127385507Z Caches written to disk