2026-09-07T12:53:11.661795347Z [disk usage checker] Started
2026-09-07T12:53:11.665270775Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T12:53:11.858873968Z Gathering data for ns/openshift-cluster-version...
2026-09-07T12:53:11.981991171Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T12:53:12.065551996Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:53:12.152945652Z Gathering data for ns/default...
2026-09-07T12:53:12.536437005Z Gathering data for ns/openshift...
2026-09-07T12:53:12.939789561Z Gathering data for ns/kube-system...
2026-09-07T12:53:13.600421418Z Gathering data for ns/openshift-etcd...
2026-09-07T12:53:13.908143698Z Wrote inspect data to must-gather.
2026-09-07T12:53:13.908176074Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T12:53:13.908176074Z     namespaces "assisted-installer" not found
2026-09-07T12:53:14.176210422Z Waiting on subprocesses to finish execution.
2026-09-07T12:53:14.183090197Z INFO: Gathering HAProxy config files
2026-09-07T12:53:14.185323660Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-07T12:53:14.187869650Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-07T12:53:14.193220663Z Executing Istio gather script
2026-09-07T12:53:14.209130719Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-07T12:53:14.212757724Z INFO: Collecting host service logs for crio
2026-09-07T12:53:14.213120673Z INFO: Collecting host service logs for kubelet
2026-09-07T12:53:14.213463615Z INFO: Collecting host service logs for rpm-ostreed
2026-09-07T12:53:14.213839503Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-07T12:53:14.214160261Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-07T12:53:14.214436731Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-07T12:53:14.214724449Z INFO: Collecting host service logs for NetworkManager
2026-09-07T12:53:14.215009125Z INFO: Collecting host service logs for openvswitch
2026-09-07T12:53:14.215301834Z INFO: Collecting host service logs for ovs-configuration
2026-09-07T12:53:14.215589011Z INFO: Collecting host service logs for ovsdb-server
2026-09-07T12:53:14.215864913Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-07T12:53:14.216203234Z INFO: Waiting for worker host service log collection to complete ...
2026-09-07T12:53:14.342427057Z INFO: Waiting for node performance related collection to complete ...
2026-09-07T12:53:14.653963815Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-07T12:53:14.700523238Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-07T12:53:14.846448166Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-07T12:53:14.944140303Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-07T12:53:14.944226866Z INFO: on-disk MachineConfig config collection complete.
2026-09-07T12:53:15.281872568Z No resources found in openshift-etcd namespace.
2026-09-07T12:53:15.306949912Z Wrote inspect data to must-gather.
2026-09-07T12:53:15.374100205Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-07T12:53:15.443297343Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-jn4hz
2026-09-07T12:53:15.443347769Z insights-runtime-extractor-cj42v
2026-09-07T12:53:15.443380195Z insights-runtime-extractor-ttdqj
2026-09-07T12:53:15.443400112Z insights-runtime-extractor-wptdf
2026-09-07T12:53:15.446735167Z No resources found
2026-09-07T12:53:15.472404180Z error: the server doesn't have a resource type "performanceprofile"
2026-09-07T12:53:15.587957615Z ERROR: No running kube-apiserver pods found
2026-09-07T12:53:15.791139212Z INFO: Worker host service log collection to complete.
2026-09-07T12:53:15.829020331Z INFO: "metallb-operator" not detected. Skipping.
2026-09-07T12:53:15.965217525Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-07T12:53:16.381032116Z Wrote inspect data to must-gather.
2026-09-07T12:53:16.406341487Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:53:16.433865352Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-07T12:53:16.480252659Z No resources found
2026-09-07T12:53:16.528235976Z Wrote inspect data to must-gather.
2026-09-07T12:53:16.530005068Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-07T12:53:16.530005068Z See 'oc rsync -h' for help and examples
2026-09-07T12:53:16.578379728Z Gathering data for ns/openshift-monitoring...
2026-09-07T12:53:16.685311763Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T12:53:16.687282990Z INFO: 'previous-logs' folder not found on ip-10-0-139-153.ec2.internal, skipping...
2026-09-07T12:53:16.733083006Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-07T12:53:16.759849675Z Wrote inspect data to must-gather.
2026-09-07T12:53:16.763339578Z Wrote inspect data to must-gather.
2026-09-07T12:53:16.765841681Z Wrote inspect data to must-gather.
2026-09-07T12:53:16.793909980Z Wrote inspect data to must-gather.
2026-09-07T12:53:16.903864589Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-07T12:53:16.945444524Z Wrote inspect data to must-gather.
2026-09-07T12:53:16.982914836Z INFO: Getting rules from prometheus-k8s-0
2026-09-07T12:53:17.023074245Z error: the server doesn't have a resource type "machineconfigs"
2026-09-07T12:53:17.049556092Z INFO: 'previous-logs' folder not found on ip-10-0-139-78.ec2.internal, skipping...
2026-09-07T12:53:17.073768594Z tar: Removing leading `/' from member names
2026-09-07T12:53:17.084264475Z INFO: HAProxy config collection complete.
2026-09-07T12:53:17.180031172Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-07T12:53:17.236669817Z Wrote inspect data to must-gather.
2026-09-07T12:53:17.308804077Z INFO: 'previous-logs' folder not found on ip-10-0-142-63.ec2.internal, skipping...
2026-09-07T12:53:17.308922297Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-07T12:53:17.308981090Z INFO: Machine Config Daemon termination log collection complete.
2026-09-07T12:53:17.414119453Z Wrote inspect data to must-gather.
2026-09-07T12:53:17.441623037Z INFO: Getting status/config from prometheus-k8s-0
2026-09-07T12:53:17.507268174Z Wrote inspect data to must-gather.
2026-09-07T12:53:17.605667825Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-07T12:53:17.833123014Z Wrote inspect data to must-gather.
2026-09-07T12:53:17.840104803Z INFO: INTERCONNECT MODE
2026-09-07T12:53:17.840145254Z INFO: Gathering ovn-kubernetes DBs
2026-09-07T12:53:17.885106109Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-07T12:53:17.997997450Z INFO: Gathering OVN_Northbound from ovnkube-node-447s5...
2026-09-07T12:53:17.998313546Z INFO: Gathering OVN_Northbound from ovnkube-node-bjzqr...
2026-09-07T12:53:17.998611725Z INFO: Gathering OVN_Northbound from ovnkube-node-gqlsr...
2026-09-07T12:53:17.998909844Z INFO: Gathering OVN_Southbound from ovnkube-node-447s5...
2026-09-07T12:53:17.999215293Z INFO: Gathering OVN_Southbound from ovnkube-node-bjzqr...
2026-09-07T12:53:17.999503666Z INFO: Gathering OVN_Southbound from ovnkube-node-gqlsr...
2026-09-07T12:53:18.066954419Z Wrote inspect data to must-gather.
2026-09-07T12:53:18.236435601Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-07T12:53:18.245023250Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-07T12:53:18.320430983Z tar: Removing leading `/' from member names
2026-09-07T12:53:18.334170513Z tar: Removing leading `/' from member names
2026-09-07T12:53:18.341223413Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T12:53:18.355616838Z tar: Removing leading `/' from member names
2026-09-07T12:53:18.381115508Z tar: Removing leading `/' from member names
2026-09-07T12:53:18.396763344Z tar: Removing leading `/' from member names
2026-09-07T12:53:18.400023716Z tar: Removing leading `/' from member names
2026-09-07T12:53:18.487596386Z Wrote inspect data to must-gather.
2026-09-07T12:53:18.494027223Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:18.551130462Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-07T12:53:18.760829894Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:18.764241886Z Wrote inspect data to must-gather.
2026-09-07T12:53:18.764594486Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-07T12:53:18.836199614Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-07T12:53:18.912628420Z INFO: Waiting for network log collection to complete ...
2026-09-07T12:53:18.912666034Z INFO: Waiting for ovnk database copies to complete ...
2026-09-07T12:53:18.913155372Z INFO: Copying ovnk databases complete.
2026-09-07T12:53:18.915327764Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-09-07T12:53:18.918252789Z ovnk_database_store/
2026-09-07T12:53:18.918252789Z ovnk_database_store/ovnkube-node-447s5_nbdb
2026-09-07T12:53:18.924679533Z ovnk_database_store/ovnkube-node-gqlsr_nbdb
2026-09-07T12:53:18.930702090Z ovnk_database_store/ovnkube-node-bjzqr_sbdb
2026-09-07T12:53:18.949730445Z ovnk_database_store/ovnkube-node-447s5_sbdb
2026-09-07T12:53:18.968167539Z ovnk_database_store/ovnkube-node-bjzqr_nbdb
2026-09-07T12:53:18.974724471Z ovnk_database_store/ovnkube-node-gqlsr_sbdb
2026-09-07T12:53:18.994214000Z INFO: Network log collection complete.
2026-09-07T12:53:19.001047668Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:19.001320561Z W0907 12:53:19.001286    1488 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-07T12:53:19.010650634Z Wrote inspect data to must-gather.
2026-09-07T12:53:19.059128519Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:53:19.243504853Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:19.246680231Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-07T12:53:19.261194693Z Gathering data for ns/openshift-network-console...
2026-09-07T12:53:19.263412999Z INFO: Getting status from alertmanager-main-0
2026-09-07T12:53:19.324985856Z Wrote inspect data to must-gather.
2026-09-07T12:53:19.448267535Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:19.632284164Z Wrote inspect data to must-gather.
2026-09-07T12:53:19.652960996Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:19.656811020Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-07T12:53:19.703107162Z Gathering data for ns/openshift-console-operator...
2026-09-07T12:53:19.880923299Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:19.887927512Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-07T12:53:19.889770490Z INFO: Fallback to identify the container image from release info
2026-09-07T12:53:20.068941682Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:20.071069658Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-07T12:53:20.204520753Z Gathering data for ns/openshift-console...
2026-09-07T12:53:20.215516697Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:20.405950513Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:53:20.415108892Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-07T12:53:20.535320131Z daemonset.apps/perf-node-gather-daemonset created
2026-09-07T12:53:20.665444402Z Waiting for performance profile collector pods to become ready: 1
2026-09-07T12:53:20.788162226Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-07T12:53:21.093873308Z Done executing Istio gather script
2026-09-07T12:53:21.341390718Z Gathering data for ns/openshift-dns-operator...
2026-09-07T12:53:21.643202102Z Gathering data for ns/openshift-dns...
2026-09-07T12:53:21.701197263Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T12:53:21.764874252Z Waiting for performance profile collector pods to become ready: 2
2026-09-07T12:53:22.375155457Z Gathering data for ns/openshift-image-registry...
2026-09-07T12:53:22.870795112Z Gathering data for ns/openshift-ingress-operator...
2026-09-07T12:53:22.884145529Z Waiting for performance profile collector pods to become ready: 3
2026-09-07T12:53:23.219147885Z Gathering data for ns/openshift-ingress...
2026-09-07T12:53:23.538286442Z Gathering data for ns/openshift-ingress-canary...
2026-09-07T12:53:23.902142147Z Gathering data for ns/openshift-insights...
2026-09-07T12:53:23.985744750Z Waiting for performance profile collector pods to become ready: 4
2026-09-07T12:53:25.084536119Z Waiting for performance profile collector pods to become ready: 5
2026-09-07T12:53:25.668974959Z Gathering data for ns/openshift-jobset-operator...
2026-09-07T12:53:26.185340630Z Waiting for performance profile collector pods to become ready: 6
2026-09-07T12:53:26.352728061Z Gathering data for ns/openshift-config...
2026-09-07T12:53:26.594541958Z Gathering data for ns/openshift-config-managed...
2026-09-07T12:53:26.706096136Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T12:53:26.978119981Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-07T12:53:27.219561408Z Gathering data for ns/openshift-kube-apiserver...
2026-09-07T12:53:27.310928757Z Waiting for performance profile collector pods to become ready: 7
2026-09-07T12:53:27.535724878Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-07T12:53:27.811665994Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-07T12:53:28.121131794Z Gathering data for ns/openshift-kube-scheduler...
2026-09-07T12:53:28.384689575Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-07T12:53:28.436640548Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-07T12:53:28.670397575Z Collecting performance related data for node ip-10-0-139-153.ec2.internal
2026-09-07T12:53:28.679405481Z Collecting performance related data for node ip-10-0-142-63.ec2.internal
2026-09-07T12:53:28.688976038Z Collecting performance related data for node ip-10-0-139-78.ec2.internal
2026-09-07T12:53:28.691510173Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-07T12:53:29.078756715Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-07T12:53:29.421735930Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-07T12:53:30.032446787Z Gathering data for ns/openshift-multus...
2026-09-07T12:53:31.534246385Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-07T12:53:31.711430393Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T12:53:32.698033199Z Gathering data for ns/openshift-host-network...
2026-09-07T12:53:32.995040460Z Gathering data for ns/openshift-network-diagnostics...
2026-09-07T12:53:33.533540455Z Gathering data for ns/openshift-network-node-identity...
2026-09-07T12:53:33.964833051Z Gathering data for ns/openshift-network-operator...
2026-09-07T12:53:34.345548738Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-07T12:53:34.640275859Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-07T12:53:35.086312387Z Gathering data for ns/openshift-apiserver-operator...
2026-09-07T12:53:35.404906601Z Gathering data for ns/openshift-apiserver...
2026-09-07T12:53:35.806730648Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-07T12:53:36.041962337Z Gathering data for ns/openshift-controller-manager...
2026-09-07T12:53:36.317472244Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-07T12:53:36.563668390Z Collecting kubelet logs for node ip-10-0-139-153.ec2.internal
2026-09-07T12:53:36.566127384Z Collecting kubelet logs for node ip-10-0-142-63.ec2.internal
2026-09-07T12:53:36.569307717Z Collecting kubelet logs for node ip-10-0-139-78.ec2.internal
2026-09-07T12:53:36.716623843Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T12:53:36.916934217Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5qvf5 namespace
2026-09-07T12:53:36.923827154Z INFO: Node performance data collection complete.
2026-09-07T12:53:37.024769723Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-07T12:53:37.296782567Z Gathering data for ns/openshift-service-ca-operator...
2026-09-07T12:53:37.592134471Z Gathering data for ns/openshift-service-ca...
2026-09-07T12:53:37.927518256Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-07T12:53:38.834048368Z Wrote inspect data to must-gather.
2026-09-07T12:53:38.834094661Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T12:53:38.834094661Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-07T12:53:38.954196661Z error: the server doesn't have a resource type "clusters"
2026-09-07T12:53:39.083912385Z Caches written to disk