2026-09-10T12:38:40.329945159Z [disk usage checker] Started
2026-09-10T12:38:40.333440053Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-10T12:38:40.541415818Z Gathering data for ns/openshift-cluster-version...
2026-09-10T12:38:40.688579504Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-10T12:38:40.788669475Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-10T12:38:40.874939292Z Gathering data for ns/default...
2026-09-10T12:38:41.200201489Z Gathering data for ns/openshift...
2026-09-10T12:38:41.693382482Z Gathering data for ns/kube-system...
2026-09-10T12:38:42.593900218Z Gathering data for ns/openshift-etcd...
2026-09-10T12:38:42.921122394Z Wrote inspect data to must-gather.
2026-09-10T12:38:42.921220143Z error: inspection completed with the errors occurred while gathering data:
2026-09-10T12:38:42.921220143Z     namespaces "assisted-installer" not found
2026-09-10T12:38:42.966986266Z Waiting on subprocesses to finish execution.
2026-09-10T12:38:42.971718279Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-10T12:38:42.972531881Z INFO: Gathering HAProxy config files
2026-09-10T12:38:42.984383066Z INFO: Collecting host service logs for crio
2026-09-10T12:38:42.984642920Z INFO: Collecting host service logs for kubelet
2026-09-10T12:38:42.984929960Z INFO: Collecting host service logs for rpm-ostreed
2026-09-10T12:38:42.985197191Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-10T12:38:42.985454335Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-10T12:38:42.985716892Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-10T12:38:42.985999320Z INFO: Collecting host service logs for NetworkManager
2026-09-10T12:38:42.986295963Z INFO: Collecting host service logs for openvswitch
2026-09-10T12:38:42.986373803Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-10T12:38:42.986616223Z INFO: Collecting host service logs for ovs-configuration
2026-09-10T12:38:42.986993520Z INFO: Collecting host service logs for ovsdb-server
2026-09-10T12:38:42.988713954Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-10T12:38:42.989155832Z INFO: Waiting for worker host service log collection to complete ...
2026-09-10T12:38:43.014464691Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-10T12:38:43.080822012Z Executing Istio gather script
2026-09-10T12:38:43.277909449Z INFO: Waiting for node performance related collection to complete ...
2026-09-10T12:38:43.646036839Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-10T12:38:43.904609944Z No resources found
2026-09-10T12:38:43.950679465Z No resources found in openshift-etcd namespace.
2026-09-10T12:38:43.968165464Z INFO: "metallb-operator" not detected. Skipping.
2026-09-10T12:38:43.977675065Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-10T12:38:43.977753820Z INFO: on-disk MachineConfig config collection complete.
2026-09-10T12:38:44.121739644Z Wrote inspect data to must-gather.
2026-09-10T12:38:44.162482509Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-10T12:38:44.167233402Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-76pvf
2026-09-10T12:38:44.167233402Z insights-runtime-extractor-7pvnm
2026-09-10T12:38:44.167233402Z insights-runtime-extractor-k2xx7
2026-09-10T12:38:44.167233402Z insights-runtime-extractor-n7gpq
2026-09-10T12:38:44.266282229Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-10T12:38:44.410980018Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-10T12:38:44.419729111Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-10T12:38:44.440659341Z Wrote inspect data to must-gather.
2026-09-10T12:38:44.479727012Z error: the server doesn't have a resource type "performanceprofile"
2026-09-10T12:38:44.481519241Z INFO: Worker host service log collection to complete.
2026-09-10T12:38:44.565542177Z ERROR: No running kube-apiserver pods found
2026-09-10T12:38:44.804979973Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-10T12:38:44.804979973Z See 'oc rsync -h' for help and examples
2026-09-10T12:38:44.912259742Z No resources found
2026-09-10T12:38:45.064515275Z Wrote inspect data to must-gather.
2026-09-10T12:38:45.116414826Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-10T12:38:45.247765469Z INFO: HAProxy config collection complete.
2026-09-10T12:38:45.287996736Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-10T12:38:45.386833300Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-10T12:38:45.404044211Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-10T12:38:45.457983623Z Wrote inspect data to must-gather.
2026-09-10T12:38:45.462586515Z Wrote inspect data to must-gather.
2026-09-10T12:38:45.473121528Z INFO: 'previous-logs' folder not found on ip-10-0-128-201.ec2.internal, skipping...
2026-09-10T12:38:45.515711025Z Wrote inspect data to must-gather.
2026-09-10T12:38:45.569275763Z Wrote inspect data to must-gather.
2026-09-10T12:38:45.649965359Z Gathering data for ns/openshift-monitoring...
2026-09-10T12:38:45.784239461Z error: the server doesn't have a resource type "machineconfigs"
2026-09-10T12:38:45.795444488Z INFO: 'previous-logs' folder not found on ip-10-0-130-132.ec2.internal, skipping...
2026-09-10T12:38:45.829788877Z Wrote inspect data to must-gather.
2026-09-10T12:38:45.861651745Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-10T12:38:45.900893046Z INFO: Getting rules from prometheus-k8s-0
2026-09-10T12:38:45.961890264Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-10T12:38:46.083477980Z INFO: 'previous-logs' folder not found on ip-10-0-130-136.ec2.internal, skipping...
2026-09-10T12:38:46.083508727Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-10T12:38:46.083516261Z INFO: Machine Config Daemon termination log collection complete.
2026-09-10T12:38:46.108808768Z Wrote inspect data to must-gather.
2026-09-10T12:38:46.208796931Z Wrote inspect data to must-gather.
2026-09-10T12:38:46.325220929Z INFO: Getting status/config from prometheus-k8s-0
2026-09-10T12:38:46.355042641Z Wrote inspect data to must-gather.
2026-09-10T12:38:46.379714294Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-10T12:38:46.593947808Z Wrote inspect data to must-gather.
2026-09-10T12:38:46.641741801Z INFO: INTERCONNECT MODE
2026-09-10T12:38:46.641781740Z INFO: Gathering ovn-kubernetes DBs
2026-09-10T12:38:46.728988768Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-10T12:38:46.807012453Z INFO: Gathering OVN_Northbound from ovnkube-node-9fsb6...
2026-09-10T12:38:46.807269367Z INFO: Gathering OVN_Northbound from ovnkube-node-fm7wp...
2026-09-10T12:38:46.807598434Z INFO: Gathering OVN_Northbound from ovnkube-node-km6pz...
2026-09-10T12:38:46.807951333Z INFO: Gathering OVN_Southbound from ovnkube-node-9fsb6...
2026-09-10T12:38:46.808245181Z INFO: Gathering OVN_Southbound from ovnkube-node-fm7wp...
2026-09-10T12:38:46.808666900Z INFO: Gathering OVN_Southbound from ovnkube-node-km6pz...
2026-09-10T12:38:46.826914220Z Wrote inspect data to must-gather.
2026-09-10T12:38:46.864003408Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-10T12:38:47.045972610Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-10T12:38:47.126036947Z tar: Removing leading `/' from member names
2026-09-10T12:38:47.126036947Z tar: Removing leading `/' from member names
2026-09-10T12:38:47.139488247Z tar: Removing leading `/' from member names
2026-09-10T12:38:47.148751488Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:47.151571718Z Wrote inspect data to must-gather.
2026-09-10T12:38:47.161367669Z tar: Removing leading `/' from member names
2026-09-10T12:38:47.193973906Z tar: Removing leading `/' from member names
2026-09-10T12:38:47.197104061Z tar: Removing leading `/' from member names
2026-09-10T12:38:47.329370719Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-10T12:38:47.415295062Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:47.417306034Z Wrote inspect data to must-gather.
2026-09-10T12:38:47.419054465Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-10T12:38:47.604009781Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-10T12:38:47.630161452Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-10T12:38:47.631854392Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:47.640953642Z Wrote inspect data to must-gather.
2026-09-10T12:38:47.645445033Z INFO: Waiting for network log collection to complete ...
2026-09-10T12:38:47.645545369Z INFO: Waiting for ovnk database copies to complete ...
2026-09-10T12:38:47.646023597Z INFO: Copying ovnk databases complete.
2026-09-10T12:38:47.647851127Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-10T12:38:47.650307409Z ovnk_database_store/
2026-09-10T12:38:47.650307409Z ovnk_database_store/ovnkube-node-fm7wp_nbdb
2026-09-10T12:38:47.659677082Z ovnk_database_store/ovnkube-node-fm7wp_sbdb
2026-09-10T12:38:47.681583731Z ovnk_database_store/ovnkube-node-km6pz_nbdb
2026-09-10T12:38:47.689245570Z ovnk_database_store/ovnkube-node-km6pz_sbdb
2026-09-10T12:38:47.710233877Z ovnk_database_store/ovnkube-node-9fsb6_sbdb
2026-09-10T12:38:47.728304723Z ovnk_database_store/ovnkube-node-9fsb6_nbdb
2026-09-10T12:38:47.740909711Z INFO: Network log collection complete.
2026-09-10T12:38:47.853998934Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:47.858048646Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-10T12:38:47.875976160Z Wrote inspect data to must-gather.
2026-09-10T12:38:48.055031380Z INFO: Getting status from alertmanager-main-0
2026-09-10T12:38:48.068086833Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:48.101579241Z Wrote inspect data to must-gather.
2026-09-10T12:38:48.270373995Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-10T12:38:48.299503266Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:48.301746829Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-10T12:38:48.371202973Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-10T12:38:48.373498884Z INFO: Fallback to identify the container image from release info
2026-09-10T12:38:48.479958228Z Gathering data for ns/openshift-network-console...
2026-09-10T12:38:48.520768864Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:48.704629551Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:48.707127312Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-10T12:38:48.876636464Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:48.936758077Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-10T12:38:48.956113138Z Gathering data for ns/openshift-console-operator...
2026-09-10T12:38:49.073803728Z daemonset.apps/perf-node-gather-daemonset created
2026-09-10T12:38:49.100232549Z Wrote inspect data to /must-gather/istio.
2026-09-10T12:38:49.195844847Z Waiting for performance profile collector pods to become ready: 1
2026-09-10T12:38:49.403148513Z Gathering data for ns/openshift-console...
2026-09-10T12:38:49.760516569Z Done executing Istio gather script
2026-09-10T12:38:49.956893986Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-10T12:38:50.317106324Z Waiting for performance profile collector pods to become ready: 2
2026-09-10T12:38:50.392010006Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-10T12:38:50.486315750Z Gathering data for ns/openshift-dns-operator...
2026-09-10T12:38:50.784949523Z Gathering data for ns/openshift-dns...
2026-09-10T12:38:51.431623254Z Waiting for performance profile collector pods to become ready: 3
2026-09-10T12:38:51.467801062Z Gathering data for ns/openshift-image-registry...
2026-09-10T12:38:51.965148123Z Gathering data for ns/openshift-ingress-operator...
2026-09-10T12:38:52.428233061Z Gathering data for ns/openshift-ingress...
2026-09-10T12:38:52.552464314Z Waiting for performance profile collector pods to become ready: 4
2026-09-10T12:38:52.796410103Z Gathering data for ns/openshift-ingress-canary...
2026-09-10T12:38:53.214702279Z Gathering data for ns/openshift-insights...
2026-09-10T12:38:53.656462939Z Waiting for performance profile collector pods to become ready: 5
2026-09-10T12:38:54.769722121Z Waiting for performance profile collector pods to become ready: 6
2026-09-10T12:38:55.192233973Z Gathering data for ns/openshift-jobset-operator...
2026-09-10T12:38:55.396641193Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-10T12:38:55.881949340Z Waiting for performance profile collector pods to become ready: 7
2026-09-10T12:38:55.998089601Z Gathering data for ns/openshift-config...
2026-09-10T12:38:56.289968167Z Gathering data for ns/openshift-config-managed...
2026-09-10T12:38:56.758038597Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-10T12:38:57.003402757Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-10T12:38:57.066086914Z Gathering data for ns/openshift-kube-apiserver...
2026-09-10T12:38:57.251619211Z Collecting performance related data for node ip-10-0-130-136.ec2.internal
2026-09-10T12:38:57.258859899Z Collecting performance related data for node ip-10-0-128-201.ec2.internal
2026-09-10T12:38:57.268632951Z Collecting performance related data for node ip-10-0-130-132.ec2.internal
2026-09-10T12:38:57.417271978Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-10T12:38:57.786163240Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-10T12:38:58.209568667Z Gathering data for ns/openshift-kube-scheduler...
2026-09-10T12:38:58.584704888Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-10T12:38:58.956830380Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-10T12:38:59.330924518Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-10T12:38:59.729236967Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-10T12:39:00.357823596Z Gathering data for ns/openshift-multus...
2026-09-10T12:39:00.401446537Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-10T12:39:01.990621068Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-10T12:39:03.443493210Z Gathering data for ns/openshift-host-network...
2026-09-10T12:39:03.776286172Z Gathering data for ns/openshift-network-diagnostics...
2026-09-10T12:39:04.378968087Z Gathering data for ns/openshift-network-node-identity...
2026-09-10T12:39:04.852283831Z Gathering data for ns/openshift-network-operator...
2026-09-10T12:39:05.280344919Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-10T12:39:05.405975495Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-10T12:39:05.422198993Z Collecting kubelet logs for node ip-10-0-130-136.ec2.internal
2026-09-10T12:39:05.423916310Z Collecting kubelet logs for node ip-10-0-128-201.ec2.internal
2026-09-10T12:39:05.427536872Z Collecting kubelet logs for node ip-10-0-130-132.ec2.internal
2026-09-10T12:39:05.609056786Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-10T12:39:05.777691724Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-6427b namespace
2026-09-10T12:39:05.785065615Z INFO: Node performance data collection complete.
2026-09-10T12:39:06.175799427Z Gathering data for ns/openshift-apiserver-operator...
2026-09-10T12:39:06.535908318Z Gathering data for ns/openshift-apiserver...
2026-09-10T12:39:06.957850114Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-10T12:39:07.294025674Z Gathering data for ns/openshift-controller-manager...
2026-09-10T12:39:07.595281926Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-10T12:39:08.363135075Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-10T12:39:08.700917216Z Gathering data for ns/openshift-service-ca-operator...
2026-09-10T12:39:09.089135670Z Gathering data for ns/openshift-service-ca...
2026-09-10T12:39:09.489758042Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-10T12:39:10.410709550Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-10T12:39:10.463690385Z Wrote inspect data to must-gather.
2026-09-10T12:39:10.463733163Z error: inspection completed with the errors occurred while gathering data:
2026-09-10T12:39:10.463733163Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-10T12:39:10.594626568Z error: the server doesn't have a resource type "clusters"
2026-09-10T12:39:10.728168642Z Caches written to disk