2026-07-24T09:09:59.997181444Z [disk usage checker] Started
2026-07-24T09:10:00.000745413Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T09:10:00.202032115Z Gathering data for ns/openshift-cluster-version...
2026-07-24T09:10:00.346965178Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T09:10:00.442204312Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T09:10:00.526481377Z Gathering data for ns/default...
2026-07-24T09:10:00.859308615Z Gathering data for ns/openshift...
2026-07-24T09:10:01.287676394Z Gathering data for ns/kube-system...
2026-07-24T09:10:01.999739922Z Gathering data for ns/openshift-etcd...
2026-07-24T09:10:02.334260604Z Wrote inspect data to must-gather.
2026-07-24T09:10:02.334295572Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T09:10:02.334295572Z     namespaces "assisted-installer" not found
2026-07-24T09:10:02.653550215Z Waiting on subprocesses to finish execution.
2026-07-24T09:10:02.665922132Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-24T09:10:02.668280656Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-24T09:10:02.670711604Z Executing Istio gather script
2026-07-24T09:10:02.680718205Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-24T09:10:02.682504010Z INFO: Gathering HAProxy config files
2026-07-24T09:10:02.695037444Z INFO: Collecting host service logs for crio
2026-07-24T09:10:02.695325187Z INFO: Collecting host service logs for kubelet
2026-07-24T09:10:02.695613682Z INFO: Collecting host service logs for rpm-ostreed
2026-07-24T09:10:02.695897625Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-24T09:10:02.696220844Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-24T09:10:02.696839858Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-24T09:10:02.697349519Z INFO: Collecting host service logs for NetworkManager
2026-07-24T09:10:02.697717004Z INFO: Collecting host service logs for openvswitch
2026-07-24T09:10:02.698085299Z INFO: Collecting host service logs for ovs-configuration
2026-07-24T09:10:02.698461807Z INFO: Collecting host service logs for ovsdb-server
2026-07-24T09:10:02.704366902Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-24T09:10:02.704786886Z INFO: Waiting for worker host service log collection to complete ...
2026-07-24T09:10:02.746850910Z INFO: Waiting for node performance related collection to complete ...
2026-07-24T09:10:03.234976389Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-bpqlp
2026-07-24T09:10:03.235034855Z insights-runtime-extractor-bkwkh
2026-07-24T09:10:03.235055578Z insights-runtime-extractor-vqvnq
2026-07-24T09:10:03.235076067Z insights-runtime-extractor-w7vq5
2026-07-24T09:10:03.340641657Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-24T09:10:03.450595783Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-24T09:10:03.482489028Z No resources found in openshift-etcd namespace.
2026-07-24T09:10:03.730734953Z INFO: "metallb-operator" not detected. Skipping.
2026-07-24T09:10:03.731354238Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-24T09:10:03.736625354Z Wrote inspect data to must-gather.
2026-07-24T09:10:03.914807439Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-24T09:10:03.914908509Z INFO: on-disk MachineConfig config collection complete.
2026-07-24T09:10:03.920168917Z error: the server doesn't have a resource type "performanceprofile"
2026-07-24T09:10:04.020483875Z No resources found
2026-07-24T09:10:04.069302094Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-24T09:10:04.069302094Z See 'oc rsync -h' for help and examples
2026-07-24T09:10:04.257266550Z INFO: Worker host service log collection to complete.
2026-07-24T09:10:04.459048688Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-24T09:10:04.533912681Z Wrote inspect data to must-gather.
2026-07-24T09:10:04.570853379Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-24T09:10:04.749994468Z Wrote inspect data to must-gather.
2026-07-24T09:10:04.753823991Z ERROR: No running kube-apiserver pods found
2026-07-24T09:10:04.817452384Z Wrote inspect data to must-gather.
2026-07-24T09:10:04.829570296Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T09:10:04.845819317Z Wrote inspect data to must-gather.
2026-07-24T09:10:04.889802689Z No resources found
2026-07-24T09:10:04.911391606Z INFO: 'previous-logs' folder not found on ip-10-0-134-169.ec2.internal, skipping...
2026-07-24T09:10:04.995087481Z Wrote inspect data to must-gather.
2026-07-24T09:10:05.023234060Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T09:10:05.088798992Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-24T09:10:05.107789886Z Wrote inspect data to must-gather.
2026-07-24T09:10:05.163703852Z error: the server doesn't have a resource type "machineconfigs"
2026-07-24T09:10:05.195389657Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-24T09:10:05.281144471Z tar: Removing leading `/' from member names
2026-07-24T09:10:05.291044123Z INFO: HAProxy config collection complete.
2026-07-24T09:10:05.385521235Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-24T09:10:05.386590355Z INFO: 'previous-logs' folder not found on ip-10-0-135-101.ec2.internal, skipping...
2026-07-24T09:10:05.463397182Z Wrote inspect data to must-gather.
2026-07-24T09:10:05.541185267Z Gathering data for ns/openshift-monitoring...
2026-07-24T09:10:05.674042394Z Wrote inspect data to must-gather.
2026-07-24T09:10:05.701962664Z INFO: Getting rules from prometheus-k8s-0
2026-07-24T09:10:05.738862837Z INFO: 'previous-logs' folder not found on ip-10-0-139-159.ec2.internal, skipping...
2026-07-24T09:10:05.738906269Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-24T09:10:05.738906269Z INFO: Machine Config Daemon termination log collection complete.
2026-07-24T09:10:05.751477492Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-24T09:10:05.809825395Z Wrote inspect data to must-gather.
2026-07-24T09:10:05.840161511Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-24T09:10:06.071226085Z Wrote inspect data to must-gather.
2026-07-24T09:10:06.073621738Z Wrote inspect data to must-gather.
2026-07-24T09:10:06.162731561Z INFO: Getting status/config from prometheus-k8s-0
2026-07-24T09:10:06.299832798Z Wrote inspect data to must-gather.
2026-07-24T09:10:06.359785970Z INFO: INTERCONNECT MODE
2026-07-24T09:10:06.359833118Z INFO: Gathering ovn-kubernetes DBs
2026-07-24T09:10:06.506501341Z Wrote inspect data to must-gather.
2026-07-24T09:10:06.528816821Z INFO: Gathering OVN_Northbound from ovnkube-node-hfps7...
2026-07-24T09:10:06.529129164Z INFO: Gathering OVN_Northbound from ovnkube-node-jqx5j...
2026-07-24T09:10:06.529368331Z INFO: Gathering OVN_Northbound from ovnkube-node-k9f4x...
2026-07-24T09:10:06.529632022Z INFO: Gathering OVN_Southbound from ovnkube-node-hfps7...
2026-07-24T09:10:06.529934333Z INFO: Gathering OVN_Southbound from ovnkube-node-jqx5j...
2026-07-24T09:10:06.530244916Z INFO: Gathering OVN_Southbound from ovnkube-node-k9f4x...
2026-07-24T09:10:06.679049992Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-24T09:10:06.818020555Z Wrote inspect data to must-gather.
2026-07-24T09:10:06.824878909Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-24T09:10:06.836897316Z tar: Removing leading `/' from member names
2026-07-24T09:10:06.838957470Z tar: Removing leading `/' from member names
2026-07-24T09:10:06.871674638Z tar: Removing leading `/' from member names
2026-07-24T09:10:06.887420877Z tar: Removing leading `/' from member names
2026-07-24T09:10:06.900069388Z tar: Removing leading `/' from member names
2026-07-24T09:10:06.912574907Z tar: Removing leading `/' from member names
2026-07-24T09:10:06.995289158Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-24T09:10:07.028363280Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:07.063182925Z Wrote inspect data to must-gather.
2026-07-24T09:10:07.278069923Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-24T09:10:07.289839650Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:07.294274617Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-24T09:10:07.321303753Z Wrote inspect data to must-gather.
2026-07-24T09:10:07.380021900Z INFO: Waiting for network log collection to complete ...
2026-07-24T09:10:07.380068516Z INFO: Waiting for ovnk database copies to complete ...
2026-07-24T09:10:07.380564614Z INFO: Copying ovnk databases complete.
2026-07-24T09:10:07.382728165Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-24T09:10:07.385270924Z ovnk_database_store/
2026-07-24T09:10:07.385270924Z ovnk_database_store/ovnkube-node-hfps7_sbdb
2026-07-24T09:10:07.406039529Z ovnk_database_store/ovnkube-node-k9f4x_sbdb
2026-07-24T09:10:07.419792291Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T09:10:07.425602785Z ovnk_database_store/ovnkube-node-k9f4x_nbdb
2026-07-24T09:10:07.431603803Z ovnk_database_store/ovnkube-node-hfps7_nbdb
2026-07-24T09:10:07.439033995Z ovnk_database_store/ovnkube-node-jqx5j_sbdb
2026-07-24T09:10:07.458660087Z ovnk_database_store/ovnkube-node-jqx5j_nbdb
2026-07-24T09:10:07.467586923Z INFO: Network log collection complete.
2026-07-24T09:10:07.516035486Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:07.550725163Z Wrote inspect data to must-gather.
2026-07-24T09:10:07.553596330Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-24T09:10:07.731212449Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:07.733553509Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-24T09:10:07.857747632Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-24T09:10:07.860727563Z INFO: Fallback to identify the container image from release info
2026-07-24T09:10:07.950209004Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:07.953781089Z INFO: Getting status from alertmanager-main-0
2026-07-24T09:10:08.064714758Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T09:10:08.165067365Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:08.167877719Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-24T09:10:08.236438632Z Gathering data for ns/openshift-network-console...
2026-07-24T09:10:08.383856030Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:08.458390723Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-24T09:10:08.599884247Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:08.602584831Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-24T09:10:08.605179283Z daemonset.apps/perf-node-gather-daemonset created
2026-07-24T09:10:08.693105535Z Gathering data for ns/openshift-console-operator...
2026-07-24T09:10:08.747819587Z Waiting for performance profile collector pods to become ready: 1
2026-07-24T09:10:08.790347622Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:08.979563380Z Wrote inspect data to /must-gather/istio.
2026-07-24T09:10:09.131715780Z Gathering data for ns/openshift-console...
2026-07-24T09:10:09.663558953Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-24T09:10:09.684480204Z Done executing Istio gather script
2026-07-24T09:10:09.859395055Z Waiting for performance profile collector pods to become ready: 2
2026-07-24T09:10:10.029964570Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T09:10:10.136435622Z Gathering data for ns/openshift-dns-operator...
2026-07-24T09:10:10.416216596Z Gathering data for ns/openshift-dns...
2026-07-24T09:10:10.984192044Z Waiting for performance profile collector pods to become ready: 3
2026-07-24T09:10:11.052223612Z Gathering data for ns/openshift-image-registry...
2026-07-24T09:10:11.560321889Z Gathering data for ns/openshift-ingress-operator...
2026-07-24T09:10:11.979857812Z Gathering data for ns/openshift-ingress...
2026-07-24T09:10:12.101190468Z Waiting for performance profile collector pods to become ready: 4
2026-07-24T09:10:12.343574478Z Gathering data for ns/openshift-ingress-canary...
2026-07-24T09:10:12.760639156Z Gathering data for ns/openshift-insights...
2026-07-24T09:10:13.232565701Z Waiting for performance profile collector pods to become ready: 5
2026-07-24T09:10:14.341845998Z Waiting for performance profile collector pods to become ready: 6
2026-07-24T09:10:14.756727232Z Gathering data for ns/openshift-jobset-operator...
2026-07-24T09:10:15.034691224Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T09:10:15.454323720Z Waiting for performance profile collector pods to become ready: 7
2026-07-24T09:10:15.562218753Z Gathering data for ns/openshift-config...
2026-07-24T09:10:15.828072561Z Gathering data for ns/openshift-config-managed...
2026-07-24T09:10:16.303262126Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-24T09:10:16.577623928Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-24T09:10:16.623642344Z Gathering data for ns/openshift-kube-apiserver...
2026-07-24T09:10:16.845371078Z Collecting performance related data for node ip-10-0-139-159.ec2.internal
2026-07-24T09:10:16.855484258Z Collecting performance related data for node ip-10-0-135-101.ec2.internal
2026-07-24T09:10:16.863953489Z Collecting performance related data for node ip-10-0-134-169.ec2.internal
2026-07-24T09:10:16.939713585Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-24T09:10:17.350328923Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-24T09:10:17.763924203Z Gathering data for ns/openshift-kube-scheduler...
2026-07-24T09:10:18.109474886Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-24T09:10:18.498378944Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-24T09:10:18.826644389Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-24T09:10:19.229755751Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-24T09:10:19.896896805Z Gathering data for ns/openshift-multus...
2026-07-24T09:10:20.039560696Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T09:10:21.470728139Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-24T09:10:22.773215636Z Gathering data for ns/openshift-host-network...
2026-07-24T09:10:23.080439747Z Gathering data for ns/openshift-network-diagnostics...
2026-07-24T09:10:23.580895565Z Gathering data for ns/openshift-network-node-identity...
2026-07-24T09:10:23.983328164Z Gathering data for ns/openshift-network-operator...
2026-07-24T09:10:24.391035681Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-24T09:10:24.663899004Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-24T09:10:25.044969616Z [disk usage checker] Volume usage percentage: current = 22 ; allowed = 70
2026-07-24T09:10:25.169591993Z Gathering data for ns/openshift-apiserver-operator...
2026-07-24T09:10:25.348657883Z Collecting kubelet logs for node ip-10-0-139-159.ec2.internal
2026-07-24T09:10:25.350576922Z Collecting kubelet logs for node ip-10-0-135-101.ec2.internal
2026-07-24T09:10:25.352519017Z Collecting kubelet logs for node ip-10-0-134-169.ec2.internal
2026-07-24T09:10:25.529621745Z Gathering data for ns/openshift-apiserver...
2026-07-24T09:10:25.723349474Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-rt75z namespace
2026-07-24T09:10:25.729802932Z INFO: Node performance data collection complete.
2026-07-24T09:10:25.953903006Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-24T09:10:26.254645367Z Gathering data for ns/openshift-controller-manager...
2026-07-24T09:10:26.520674825Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-24T09:10:27.153245634Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-24T09:10:27.444931941Z Gathering data for ns/openshift-service-ca-operator...
2026-07-24T09:10:27.806868143Z Gathering data for ns/openshift-service-ca...
2026-07-24T09:10:28.204046156Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-24T09:10:29.118157496Z Wrote inspect data to must-gather.
2026-07-24T09:10:29.118208383Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T09:10:29.118208383Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-24T09:10:29.257534170Z error: the server doesn't have a resource type "clusters"
2026-07-24T09:10:29.427211388Z Caches written to disk