2026-09-11T06:29:20.762056746Z [disk usage checker] Started 2026-09-11T06:29:20.765392622Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-11T06:29:20.957077644Z Gathering data for ns/openshift-cluster-version... 2026-09-11T06:29:21.080997722Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-11T06:29:21.158071174Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-11T06:29:21.236697150Z Gathering data for ns/default... 2026-09-11T06:29:21.537141028Z Gathering data for ns/openshift... 2026-09-11T06:29:21.954894901Z Gathering data for ns/kube-system... 2026-09-11T06:29:22.591556845Z Gathering data for ns/openshift-etcd... 2026-09-11T06:29:22.890716737Z Wrote inspect data to must-gather. 2026-09-11T06:29:22.890847443Z error: inspection completed with the errors occurred while gathering data: 2026-09-11T06:29:22.890847443Z namespaces "assisted-installer" not found 2026-09-11T06:29:23.351213763Z Waiting on subprocesses to finish execution. 2026-09-11T06:29:23.355337259Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-11T06:29:23.355947078Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-11T06:29:23.367495810Z INFO: Gathering HAProxy config files 2026-09-11T06:29:23.383371133Z Executing Istio gather script 2026-09-11T06:29:23.384142633Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-11T06:29:23.390195021Z INFO: Collecting host service logs for crio 2026-09-11T06:29:23.390507754Z INFO: Collecting host service logs for kubelet 2026-09-11T06:29:23.390855712Z INFO: Collecting host service logs for rpm-ostreed 2026-09-11T06:29:23.391164201Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-11T06:29:23.391468078Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-11T06:29:23.391887226Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-11T06:29:23.392206378Z INFO: Collecting host service logs for NetworkManager 2026-09-11T06:29:23.392495130Z INFO: Collecting host service logs for openvswitch 2026-09-11T06:29:23.392848902Z INFO: Collecting host service logs for ovs-configuration 2026-09-11T06:29:23.396558727Z INFO: Collecting host service logs for ovsdb-server 2026-09-11T06:29:23.396558727Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-11T06:29:23.396558727Z INFO: Waiting for worker host service log collection to complete ... 2026-09-11T06:29:23.399777358Z INFO: Waiting for node performance related collection to complete ... 2026-09-11T06:29:24.006419062Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-11T06:29:24.006457981Z INFO: on-disk MachineConfig config collection complete. 2026-09-11T06:29:24.054516396Z INFO: Collecting Insights Archives from insights-operator-5844b856db-kcmwr 2026-09-11T06:29:24.054516396Z insights-runtime-extractor-6r98r 2026-09-11T06:29:24.054516396Z insights-runtime-extractor-7c5w7 2026-09-11T06:29:24.054516396Z insights-runtime-extractor-864w2 2026-09-11T06:29:24.224290026Z INFO: "metallb-operator" not detected. Skipping. 2026-09-11T06:29:24.515202425Z No resources found 2026-09-11T06:29:24.586017385Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-11T06:29:24.609140240Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-11T06:29:24.823006323Z No resources found in openshift-etcd namespace. 2026-09-11T06:29:24.827535548Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-11T06:29:24.852001730Z error: the server doesn't have a resource type "performanceprofile" 2026-09-11T06:29:24.937410000Z Wrote inspect data to must-gather. 2026-09-11T06:29:25.054055601Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-11T06:29:25.168271514Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-11T06:29:25.331600329Z Wrote inspect data to must-gather. 2026-09-11T06:29:25.437387043Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-11T06:29:25.437387043Z See 'oc rsync -h' for help and examples 2026-09-11T06:29:25.551225906Z No resources found 2026-09-11T06:29:25.562672806Z INFO: Worker host service log collection to complete. 2026-09-11T06:29:25.579094895Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-11T06:29:25.580736596Z Wrote inspect data to must-gather. 2026-09-11T06:29:25.586829279Z ERROR: No running kube-apiserver pods found 2026-09-11T06:29:25.665080770Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-11T06:29:25.679586474Z Wrote inspect data to must-gather. 2026-09-11T06:29:25.706964094Z Wrote inspect data to must-gather. 2026-09-11T06:29:25.740606058Z Wrote inspect data to must-gather. 2026-09-11T06:29:25.780841837Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-11T06:29:25.781990451Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-11T06:29:25.783213797Z INFO: 'previous-logs' folder not found on ip-10-0-135-168.ec2.internal, skipping... 2026-09-11T06:29:25.823761026Z Wrote inspect data to must-gather. 2026-09-11T06:29:26.018935132Z tar: Removing leading `/' from member names 2026-09-11T06:29:26.019197903Z error: the server doesn't have a resource type "machineconfigs" 2026-09-11T06:29:26.023528145Z INFO: HAProxy config collection complete. 2026-09-11T06:29:26.036062061Z Wrote inspect data to must-gather. 2026-09-11T06:29:26.170596954Z INFO: 'previous-logs' folder not found on ip-10-0-136-46.ec2.internal, skipping... 2026-09-11T06:29:26.225977196Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-11T06:29:26.260958430Z Gathering data for ns/openshift-monitoring... 2026-09-11T06:29:26.342385990Z Wrote inspect data to must-gather. 2026-09-11T06:29:26.346939840Z INFO: Getting rules from prometheus-k8s-0 2026-09-11T06:29:26.481242116Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-11T06:29:26.499927195Z Wrote inspect data to must-gather. 2026-09-11T06:29:26.504742580Z INFO: 'previous-logs' folder not found on ip-10-0-142-229.ec2.internal, skipping... 2026-09-11T06:29:26.504841189Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-11T06:29:26.504890250Z INFO: Machine Config Daemon termination log collection complete. 2026-09-11T06:29:26.633487227Z Wrote inspect data to must-gather. 2026-09-11T06:29:26.667409559Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-11T06:29:26.759312508Z INFO: Getting status/config from prometheus-k8s-0 2026-09-11T06:29:26.906354195Z Wrote inspect data to must-gather. 2026-09-11T06:29:26.918341834Z INFO: INTERCONNECT MODE 2026-09-11T06:29:26.918367841Z INFO: Gathering ovn-kubernetes DBs 2026-09-11T06:29:27.078383650Z INFO: Gathering OVN_Northbound from ovnkube-node-7ztw9... 2026-09-11T06:29:27.078620150Z INFO: Gathering OVN_Northbound from ovnkube-node-gkzwh... 2026-09-11T06:29:27.078826024Z INFO: Gathering OVN_Northbound from ovnkube-node-hrh7k... 2026-09-11T06:29:27.079111730Z INFO: Gathering OVN_Southbound from ovnkube-node-7ztw9... 2026-09-11T06:29:27.079562502Z INFO: Gathering OVN_Southbound from ovnkube-node-gkzwh... 2026-09-11T06:29:27.079895704Z INFO: Gathering OVN_Southbound from ovnkube-node-hrh7k... 2026-09-11T06:29:27.182438278Z Wrote inspect data to must-gather. 2026-09-11T06:29:27.317746989Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-11T06:29:27.383025209Z tar: Removing leading `/' from member names 2026-09-11T06:29:27.383129957Z tar: Removing leading `/' from member names 2026-09-11T06:29:27.419236610Z tar: Removing leading `/' from member names 2026-09-11T06:29:27.450080430Z tar: Removing leading `/' from member names 2026-09-11T06:29:27.450365569Z tar: Removing leading `/' from member names 2026-09-11T06:29:27.456090468Z tar: Removing leading `/' from member names 2026-09-11T06:29:27.537266012Z Wrote inspect data to must-gather. 2026-09-11T06:29:27.541042392Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-11T06:29:27.601971792Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-11T06:29:27.785268175Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:27.806689190Z Wrote inspect data to must-gather. 2026-09-11T06:29:27.864972269Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-11T06:29:27.872103094Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-11T06:29:27.892224689Z INFO: Waiting for network log collection to complete ... 2026-09-11T06:29:27.892331624Z INFO: Waiting for ovnk database copies to complete ... 2026-09-11T06:29:27.892844946Z INFO: Copying ovnk databases complete. 2026-09-11T06:29:27.894957344Z 2.8M must-gather/network_logs/ovnk_database_store 2026-09-11T06:29:27.897966038Z ovnk_database_store/ 2026-09-11T06:29:27.898013756Z ovnk_database_store/ovnkube-node-7ztw9_nbdb 2026-09-11T06:29:27.906950319Z ovnk_database_store/ovnkube-node-7ztw9_sbdb 2026-09-11T06:29:27.925740465Z ovnk_database_store/ovnkube-node-hrh7k_sbdb 2026-09-11T06:29:27.946805433Z ovnk_database_store/ovnkube-node-gkzwh_sbdb 2026-09-11T06:29:27.965615886Z ovnk_database_store/ovnkube-node-hrh7k_nbdb 2026-09-11T06:29:27.973978255Z ovnk_database_store/ovnkube-node-gkzwh_nbdb 2026-09-11T06:29:27.984864658Z INFO: Network log collection complete. 2026-09-11T06:29:28.014525310Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:28.016927252Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-11T06:29:28.028629967Z Wrote inspect data to must-gather. 2026-09-11T06:29:28.125809222Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-11T06:29:28.230294875Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:28.246102295Z Wrote inspect data to must-gather. 2026-09-11T06:29:28.447471416Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:28.449836146Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-11T06:29:28.460064580Z W0911 06:29:28.460027 1521 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-11T06:29:28.466120666Z Wrote inspect data to must-gather. 2026-09-11T06:29:28.501414853Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-11T06:29:28.519336864Z INFO: Getting status from alertmanager-main-0 2026-09-11T06:29:28.671356306Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:28.708982833Z Gathering data for ns/openshift-network-console... 2026-09-11T06:29:28.734686638Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-11T06:29:28.736533737Z INFO: Fallback to identify the container image from release info 2026-09-11T06:29:28.874411776Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:28.877039564Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-11T06:29:29.100538969Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:29.113214293Z Gathering data for ns/openshift-console-operator... 2026-09-11T06:29:29.300604038Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:29.304448200Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-11T06:29:29.428364192Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-11T06:29:29.478632995Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:29.506617015Z Gathering data for ns/openshift-console... 2026-09-11T06:29:29.563869986Z daemonset.apps/perf-node-gather-daemonset created 2026-09-11T06:29:29.687187522Z Wrote inspect data to /must-gather/istio. 2026-09-11T06:29:29.712446962Z Waiting for performance profile collector pods to become ready: 1 2026-09-11T06:29:30.045197751Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-11T06:29:30.370926704Z Done executing Istio gather script 2026-09-11T06:29:30.520244662Z Gathering data for ns/openshift-dns-operator... 2026-09-11T06:29:30.764433765Z Gathering data for ns/openshift-dns... 2026-09-11T06:29:30.788350392Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-11T06:29:30.829606680Z Waiting for performance profile collector pods to become ready: 2 2026-09-11T06:29:31.286199717Z Gathering data for ns/openshift-image-registry... 2026-09-11T06:29:31.690140782Z Gathering data for ns/openshift-ingress-operator... 2026-09-11T06:29:31.954961891Z Waiting for performance profile collector pods to become ready: 3 2026-09-11T06:29:31.993242711Z Gathering data for ns/openshift-ingress... 2026-09-11T06:29:32.278144636Z Gathering data for ns/openshift-ingress-canary... 2026-09-11T06:29:32.613824326Z Gathering data for ns/openshift-insights... 2026-09-11T06:29:33.070288648Z Waiting for performance profile collector pods to become ready: 4 2026-09-11T06:29:34.172608133Z Waiting for performance profile collector pods to become ready: 5 2026-09-11T06:29:34.379642098Z Gathering data for ns/openshift-jobset-operator... 2026-09-11T06:29:35.143666663Z Gathering data for ns/openshift-config... 2026-09-11T06:29:35.281588689Z Waiting for performance profile collector pods to become ready: 6 2026-09-11T06:29:35.504443670Z Gathering data for ns/openshift-config-managed... 2026-09-11T06:29:35.795190962Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-11T06:29:35.919722576Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-11T06:29:36.191456099Z Gathering data for ns/openshift-kube-apiserver... 2026-09-11T06:29:36.406907467Z Waiting for performance profile collector pods to become ready: 7 2026-09-11T06:29:36.432384304Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-11T06:29:36.694204513Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-11T06:29:36.945026550Z Gathering data for ns/openshift-kube-scheduler... 2026-09-11T06:29:37.178329161Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-11T06:29:37.481724125Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-11T06:29:37.521735485Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-11T06:29:37.783054337Z Collecting performance related data for node ip-10-0-142-229.ec2.internal 2026-09-11T06:29:37.792586734Z Collecting performance related data for node ip-10-0-136-46.ec2.internal 2026-09-11T06:29:37.799198260Z Collecting performance related data for node ip-10-0-135-168.ec2.internal 2026-09-11T06:29:37.799514553Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-11T06:29:38.169149551Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-11T06:29:38.758592570Z Gathering data for ns/openshift-multus... 2026-09-11T06:29:40.126839167Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-11T06:29:40.800317934Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T06:29:41.270133703Z Gathering data for ns/openshift-host-network... 2026-09-11T06:29:41.590764565Z Gathering data for ns/openshift-network-diagnostics... 2026-09-11T06:29:42.118223713Z Gathering data for ns/openshift-network-node-identity... 2026-09-11T06:29:42.530136437Z Gathering data for ns/openshift-network-operator... 2026-09-11T06:29:42.889138474Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-11T06:29:43.151898383Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-11T06:29:43.599692555Z Gathering data for ns/openshift-apiserver-operator... 2026-09-11T06:29:43.876608184Z Gathering data for ns/openshift-apiserver... 2026-09-11T06:29:44.243241553Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-11T06:29:44.515893760Z Gathering data for ns/openshift-controller-manager... 2026-09-11T06:29:44.758800754Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-11T06:29:45.389622216Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-11T06:29:45.540004559Z Collecting kubelet logs for node ip-10-0-142-229.ec2.internal 2026-09-11T06:29:45.542293578Z Collecting kubelet logs for node ip-10-0-136-46.ec2.internal 2026-09-11T06:29:45.545787025Z Collecting kubelet logs for node ip-10-0-135-168.ec2.internal 2026-09-11T06:29:45.716453122Z Gathering data for ns/openshift-service-ca-operator... 2026-09-11T06:29:45.807250721Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T06:29:45.904363135Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-q2stp namespace 2026-09-11T06:29:45.909115370Z INFO: Node performance data collection complete. 2026-09-11T06:29:46.027355172Z Gathering data for ns/openshift-service-ca... 2026-09-11T06:29:46.381020200Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-11T06:29:47.207470632Z Wrote inspect data to must-gather. 2026-09-11T06:29:47.207515521Z error: inspection completed with the errors occurred while gathering data: 2026-09-11T06:29:47.207515521Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-11T06:29:47.340311413Z error: the server doesn't have a resource type "clusters" 2026-09-11T06:29:47.475492754Z Caches written to disk