2026-06-05T16:57:15.836536613Z [disk usage checker] Started
2026-06-05T16:57:15.840198953Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-05T16:57:16.026641936Z Gathering data for ns/openshift-cluster-version...
2026-06-05T16:57:16.168771110Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T16:57:16.254467952Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T16:57:16.337770730Z Gathering data for ns/default...
2026-06-05T16:57:16.627270290Z Gathering data for ns/openshift...
2026-06-05T16:57:17.013242436Z Gathering data for ns/kube-system...
2026-06-05T16:57:17.669253634Z Gathering data for ns/openshift-etcd...
2026-06-05T16:57:17.985486808Z Wrote inspect data to must-gather.
2026-06-05T16:57:17.985528279Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T16:57:17.985528279Z     namespaces "assisted-installer" not found
2026-06-05T16:57:18.336016841Z Waiting on subprocesses to finish execution.
2026-06-05T16:57:18.345878085Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-05T16:57:18.348263487Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-05T16:57:18.348894915Z Executing Istio gather script
2026-06-05T16:57:18.354066710Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-05T16:57:18.355728100Z INFO: Gathering HAProxy config files
2026-06-05T16:57:18.369899039Z INFO: Collecting host service logs for crio
2026-06-05T16:57:18.370283688Z INFO: Collecting host service logs for kubelet
2026-06-05T16:57:18.370607729Z INFO: Collecting host service logs for rpm-ostreed
2026-06-05T16:57:18.370932978Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-05T16:57:18.371260429Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-05T16:57:18.371599701Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-05T16:57:18.372092619Z INFO: Collecting host service logs for NetworkManager
2026-06-05T16:57:18.372429745Z INFO: Collecting host service logs for openvswitch
2026-06-05T16:57:18.372757813Z INFO: Collecting host service logs for ovs-configuration
2026-06-05T16:57:18.373097100Z INFO: Collecting host service logs for ovsdb-server
2026-06-05T16:57:18.373454321Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-05T16:57:18.373848962Z INFO: Waiting for worker host service log collection to complete ...
2026-06-05T16:57:18.396908254Z INFO: Waiting for node performance related collection to complete ...
2026-06-05T16:57:18.844887291Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-05T16:57:18.928228788Z No resources found
2026-06-05T16:57:18.957558994Z No resources found in openshift-etcd namespace.
2026-06-05T16:57:19.175928500Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-05T16:57:19.332551274Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-05T16:57:19.344487079Z INFO: Collecting Insights Archives from insights-operator-896549976-r886d
2026-06-05T16:57:19.344537667Z insights-runtime-extractor-5rtmh
2026-06-05T16:57:19.344556730Z insights-runtime-extractor-dfdjt
2026-06-05T16:57:19.344573377Z insights-runtime-extractor-mjp5q
2026-06-05T16:57:19.601516121Z INFO: "metallb-operator" not detected. Skipping.
2026-06-05T16:57:19.605836756Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-05T16:57:19.622149026Z error: the server doesn't have a resource type "performanceprofile"
2026-06-05T16:57:19.642852128Z INFO: Worker host service log collection to complete.
2026-06-05T16:57:19.668021020Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-05T16:57:19.668060732Z INFO: on-disk MachineConfig config collection complete.
2026-06-05T16:57:19.806633294Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-05T16:57:19.806633294Z See 'oc rsync -h' for help and examples
2026-06-05T16:57:19.864342016Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-05T16:57:19.923205863Z Wrote inspect data to must-gather.
2026-06-05T16:57:19.981816789Z ERROR: No running kube-apiserver pods found
2026-06-05T16:57:20.074803020Z Gathering data for ns/openshift-monitoring...
2026-06-05T16:57:20.219642510Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-05T16:57:20.346347438Z Wrote inspect data to must-gather.
2026-06-05T16:57:20.425241010Z Wrote inspect data to must-gather.
2026-06-05T16:57:20.476574175Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T16:57:20.517966254Z tar: Removing leading `/' from member names
2026-06-05T16:57:20.522849478Z INFO: HAProxy config collection complete.
2026-06-05T16:57:20.885532141Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-05T16:57:20.907094671Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-05T16:57:20.917991721Z No resources found
2026-06-05T16:57:20.927438964Z error: the server doesn't have a resource type "machineconfigs"
2026-06-05T16:57:20.930148131Z Wrote inspect data to must-gather.
2026-06-05T16:57:20.953012422Z Wrote inspect data to must-gather.
2026-06-05T16:57:20.981773857Z Wrote inspect data to must-gather.
2026-06-05T16:57:21.016029943Z INFO: 'previous-logs' folder not found on ip-10-0-130-218.ec2.internal, skipping...
2026-06-05T16:57:21.028438907Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-05T16:57:21.105047616Z Wrote inspect data to must-gather.
2026-06-05T16:57:21.135387054Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-05T16:57:21.221095517Z INFO: Getting rules from prometheus-k8s-0
2026-06-05T16:57:21.242388422Z Wrote inspect data to must-gather.
2026-06-05T16:57:21.295982114Z INFO: 'previous-logs' folder not found on ip-10-0-134-160.ec2.internal, skipping...
2026-06-05T16:57:21.369439802Z Wrote inspect data to must-gather.
2026-06-05T16:57:21.499540106Z Wrote inspect data to must-gather.
2026-06-05T16:57:21.541895383Z INFO: 'previous-logs' folder not found on ip-10-0-143-99.ec2.internal, skipping...
2026-06-05T16:57:21.542001324Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-05T16:57:21.542053036Z INFO: Machine Config Daemon termination log collection complete.
2026-06-05T16:57:21.551787598Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-05T16:57:21.613735140Z INFO: Getting status/config from prometheus-k8s-0
2026-06-05T16:57:21.779549868Z Wrote inspect data to must-gather.
2026-06-05T16:57:21.787918091Z Wrote inspect data to must-gather.
2026-06-05T16:57:21.907833688Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T16:57:21.994271854Z Wrote inspect data to must-gather.
2026-06-05T16:57:22.045342614Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-05T16:57:22.069125656Z INFO: INTERCONNECT MODE
2026-06-05T16:57:22.069125656Z INFO: Gathering ovn-kubernetes DBs
2026-06-05T16:57:22.209002454Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-05T16:57:22.229381512Z Wrote inspect data to must-gather.
2026-06-05T16:57:22.237829840Z INFO: Gathering OVN_Northbound from ovnkube-node-4v7rc...
2026-06-05T16:57:22.238047820Z INFO: Gathering OVN_Northbound from ovnkube-node-lszzx...
2026-06-05T16:57:22.238345841Z INFO: Gathering OVN_Northbound from ovnkube-node-sxl8f...
2026-06-05T16:57:22.238590946Z INFO: Gathering OVN_Southbound from ovnkube-node-4v7rc...
2026-06-05T16:57:22.238890051Z INFO: Gathering OVN_Southbound from ovnkube-node-lszzx...
2026-06-05T16:57:22.239143833Z INFO: Gathering OVN_Southbound from ovnkube-node-sxl8f...
2026-06-05T16:57:22.309975142Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-05T16:57:22.524715657Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:22.579855875Z tar: Removing leading `/' from member names
2026-06-05T16:57:22.590190068Z tar: Removing leading `/' from member names
2026-06-05T16:57:22.591443301Z tar: Removing leading `/' from member names
2026-06-05T16:57:22.613204578Z tar: Removing leading `/' from member names
2026-06-05T16:57:22.640750369Z W0605 16:57:22.640704    1329 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-05T16:57:22.643183457Z tar: Removing leading `/' from member names
2026-06-05T16:57:22.647165625Z Wrote inspect data to must-gather.
2026-06-05T16:57:22.666193768Z tar: Removing leading `/' from member names
2026-06-05T16:57:22.719460549Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-05T16:57:22.724088629Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T16:57:22.784085994Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:22.786560064Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-05T16:57:22.970855845Z Gathering data for ns/openshift-network-console...
2026-06-05T16:57:22.996693912Z Wrote inspect data to must-gather.
2026-06-05T16:57:23.004760496Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-05T16:57:23.019945131Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:23.113559027Z INFO: Waiting for network log collection to complete ...
2026-06-05T16:57:23.113674697Z INFO: Waiting for ovnk database copies to complete ...
2026-06-05T16:57:23.114217481Z INFO: Copying ovnk databases complete.
2026-06-05T16:57:23.116244086Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-06-05T16:57:23.121486475Z ovnk_database_store/
2026-06-05T16:57:23.121534500Z ovnk_database_store/ovnkube-node-sxl8f_nbdb
2026-06-05T16:57:23.131430129Z ovnk_database_store/ovnkube-node-lszzx_nbdb
2026-06-05T16:57:23.140251756Z ovnk_database_store/ovnkube-node-lszzx_sbdb
2026-06-05T16:57:23.159518355Z ovnk_database_store/ovnkube-node-sxl8f_sbdb
2026-06-05T16:57:23.175799290Z ovnk_database_store/ovnkube-node-4v7rc_nbdb
2026-06-05T16:57:23.183609753Z ovnk_database_store/ovnkube-node-4v7rc_sbdb
2026-06-05T16:57:23.203570511Z INFO: Network log collection complete.
2026-06-05T16:57:23.267264485Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:23.274899551Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-05T16:57:23.315972222Z Wrote inspect data to must-gather.
2026-06-05T16:57:23.416033175Z INFO: Getting status from alertmanager-main-0
2026-06-05T16:57:23.490707378Z Gathering data for ns/openshift-console-operator...
2026-06-05T16:57:23.509335355Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:23.637206287Z Wrote inspect data to must-gather.
2026-06-05T16:57:23.713214386Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:23.715349129Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-05T16:57:23.898661944Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-05T16:57:23.901694415Z INFO: Fallback to identify the container image from release info
2026-06-05T16:57:23.908371734Z Gathering data for ns/openshift-console...
2026-06-05T16:57:23.945182513Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:24.127629984Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:24.130429579Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-05T16:57:24.292427479Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:24.395270485Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-05T16:57:24.496561829Z Wrote inspect data to /must-gather/istio.
2026-06-05T16:57:24.511324753Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-05T16:57:24.641863859Z daemonset.apps/perf-node-gather-daemonset created
2026-06-05T16:57:24.780905494Z Waiting for performance profile collector pods to become ready: 1
2026-06-05T16:57:24.940370014Z Gathering data for ns/openshift-dns-operator...
2026-06-05T16:57:25.215219246Z Done executing Istio gather script
2026-06-05T16:57:25.247456397Z Gathering data for ns/openshift-dns...
2026-06-05T16:57:25.834144968Z Gathering data for ns/openshift-image-registry...
2026-06-05T16:57:25.894048957Z Waiting for performance profile collector pods to become ready: 2
2026-06-05T16:57:25.901467201Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-05T16:57:26.295665492Z Gathering data for ns/openshift-ingress-operator...
2026-06-05T16:57:26.611196545Z Gathering data for ns/openshift-ingress...
2026-06-05T16:57:26.941916644Z Gathering data for ns/openshift-ingress-canary...
2026-06-05T16:57:27.002956700Z Waiting for performance profile collector pods to become ready: 3
2026-06-05T16:57:27.332091177Z Gathering data for ns/openshift-insights...
2026-06-05T16:57:28.108254565Z Waiting for performance profile collector pods to become ready: 4
2026-06-05T16:57:29.177158457Z Gathering data for ns/openshift-jobset-operator...
2026-06-05T16:57:29.218495053Z Waiting for performance profile collector pods to become ready: 5
2026-06-05T16:57:29.946899574Z Gathering data for ns/openshift-config...
2026-06-05T16:57:30.212349853Z Gathering data for ns/openshift-config-managed...
2026-06-05T16:57:30.338778224Z Waiting for performance profile collector pods to become ready: 6
2026-06-05T16:57:30.635238365Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-05T16:57:30.877508622Z Gathering data for ns/openshift-kube-apiserver...
2026-06-05T16:57:30.907769873Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-05T16:57:31.152488305Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-05T16:57:31.452111326Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-05T16:57:31.458155623Z Waiting for performance profile collector pods to become ready: 7
2026-06-05T16:57:31.755709958Z Gathering data for ns/openshift-kube-scheduler...
2026-06-05T16:57:32.013802027Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-05T16:57:32.295764180Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-05T16:57:32.573235716Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-05T16:57:32.647729071Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-05T16:57:32.809954876Z Collecting performance related data for node ip-10-0-143-99.ec2.internal
2026-06-05T16:57:32.818118788Z Collecting performance related data for node ip-10-0-130-218.ec2.internal
2026-06-05T16:57:32.829682388Z Collecting performance related data for node ip-10-0-134-160.ec2.internal
2026-06-05T16:57:33.005664848Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-05T16:57:33.560150130Z Gathering data for ns/openshift-multus...
2026-06-05T16:57:35.038374933Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-05T16:57:35.912789671Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-05T16:57:36.263435827Z Gathering data for ns/openshift-host-network...
2026-06-05T16:57:36.634269441Z Gathering data for ns/openshift-network-diagnostics...
2026-06-05T16:57:37.174716216Z Gathering data for ns/openshift-network-node-identity...
2026-06-05T16:57:37.579933704Z Gathering data for ns/openshift-network-operator...
2026-06-05T16:57:37.963593360Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-05T16:57:38.215390198Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-05T16:57:38.673664764Z Gathering data for ns/openshift-apiserver-operator...
2026-06-05T16:57:38.962438237Z Gathering data for ns/openshift-apiserver...
2026-06-05T16:57:39.432103038Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-05T16:57:39.720732417Z Gathering data for ns/openshift-controller-manager...
2026-06-05T16:57:39.988804007Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-05T16:57:40.295838070Z Collecting kubelet logs for node ip-10-0-143-99.ec2.internal
2026-06-05T16:57:40.297955538Z Collecting kubelet logs for node ip-10-0-130-218.ec2.internal
2026-06-05T16:57:40.300211475Z Collecting kubelet logs for node ip-10-0-134-160.ec2.internal
2026-06-05T16:57:40.623689376Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-2hfnq namespace
2026-06-05T16:57:40.628064170Z INFO: Node performance data collection complete.
2026-06-05T16:57:40.715109007Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-05T16:57:40.918509657Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-05T16:57:40.994380294Z Gathering data for ns/openshift-service-ca-operator...
2026-06-05T16:57:41.303365944Z Gathering data for ns/openshift-service-ca...
2026-06-05T16:57:41.649726683Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-05T16:57:42.501870005Z Wrote inspect data to must-gather.
2026-06-05T16:57:42.501912912Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T16:57:42.501912912Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-05T16:57:42.615214705Z error: the server doesn't have a resource type "clusters"
2026-06-05T16:57:42.750805552Z Caches written to disk