2026-07-29T11:35:45.344509156Z [disk usage checker] Started
2026-07-29T11:35:45.347960802Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-29T11:35:45.551704100Z Gathering data for ns/openshift-cluster-version...
2026-07-29T11:35:45.688639417Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T11:35:45.767014359Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T11:35:45.849301491Z Gathering data for ns/default...
2026-07-29T11:35:46.122567719Z Gathering data for ns/openshift...
2026-07-29T11:35:46.509244302Z Gathering data for ns/kube-system...
2026-07-29T11:35:47.144486191Z Gathering data for ns/openshift-etcd...
2026-07-29T11:35:47.440793141Z Wrote inspect data to must-gather.
2026-07-29T11:35:47.440948041Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T11:35:47.440948041Z     namespaces "assisted-installer" not found
2026-07-29T11:35:47.902713535Z Waiting on subprocesses to finish execution.
2026-07-29T11:35:47.908076465Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-29T11:35:47.911931673Z Executing Istio gather script
2026-07-29T11:35:47.915706018Z INFO: Gathering HAProxy config files
2026-07-29T11:35:47.927448586Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-29T11:35:47.928189971Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-29T11:35:47.948510410Z INFO: Collecting host service logs for crio
2026-07-29T11:35:47.948792428Z INFO: Collecting host service logs for kubelet
2026-07-29T11:35:47.949127691Z INFO: Collecting host service logs for rpm-ostreed
2026-07-29T11:35:47.949416667Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-29T11:35:47.949708678Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-29T11:35:47.950020991Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-29T11:35:47.950297803Z INFO: Collecting host service logs for NetworkManager
2026-07-29T11:35:47.950596328Z INFO: Collecting host service logs for openvswitch
2026-07-29T11:35:47.950887892Z INFO: Collecting host service logs for ovs-configuration
2026-07-29T11:35:47.951291021Z INFO: Collecting host service logs for ovsdb-server
2026-07-29T11:35:47.951587318Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-29T11:35:47.951947302Z INFO: Waiting for worker host service log collection to complete ...
2026-07-29T11:35:48.023310961Z INFO: Waiting for node performance related collection to complete ...
2026-07-29T11:35:48.382809373Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-29T11:35:48.481660373Z INFO: "metallb-operator" not detected. Skipping.
2026-07-29T11:35:48.613989230Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-29T11:35:48.614129556Z INFO: on-disk MachineConfig config collection complete.
2026-07-29T11:35:48.688526474Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-fhrsv
2026-07-29T11:35:48.688585938Z insights-runtime-extractor-4pcb8
2026-07-29T11:35:48.688607468Z insights-runtime-extractor-78cqw
2026-07-29T11:35:48.688627088Z insights-runtime-extractor-rvgln
2026-07-29T11:35:48.745770846Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-29T11:35:48.781468490Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-29T11:35:48.845804590Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-29T11:35:48.962977913Z No resources found in openshift-etcd namespace.
2026-07-29T11:35:49.024699249Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-29T11:35:49.024699249Z See 'oc rsync -h' for help and examples
2026-07-29T11:35:49.197035377Z error: the server doesn't have a resource type "performanceprofile"
2026-07-29T11:35:49.245698494Z No resources found
2026-07-29T11:35:49.286722631Z Wrote inspect data to must-gather.
2026-07-29T11:35:49.300201782Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-29T11:35:49.319280722Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-29T11:35:49.426629863Z INFO: Worker host service log collection to complete.
2026-07-29T11:35:49.558109115Z ERROR: No running kube-apiserver pods found
2026-07-29T11:35:49.718515633Z tar: Removing leading `/' from member names
2026-07-29T11:35:49.726391875Z INFO: HAProxy config collection complete.
2026-07-29T11:35:49.785190561Z Wrote inspect data to must-gather.
2026-07-29T11:35:49.936317155Z Wrote inspect data to must-gather.
2026-07-29T11:35:50.089090439Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T11:35:50.219398765Z INFO: 'previous-logs' folder not found on ip-10-0-131-5.ec2.internal, skipping...
2026-07-29T11:35:50.272283027Z Gathering data for ns/openshift-monitoring...
2026-07-29T11:35:50.386551540Z No resources found
2026-07-29T11:35:50.407626928Z Wrote inspect data to must-gather.
2026-07-29T11:35:50.411655111Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-29T11:35:50.415481163Z Wrote inspect data to must-gather.
2026-07-29T11:35:50.416873516Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-29T11:35:50.469994974Z Wrote inspect data to must-gather.
2026-07-29T11:35:50.523178670Z INFO: Getting rules from prometheus-k8s-0
2026-07-29T11:35:50.565505808Z error: the server doesn't have a resource type "machineconfigs"
2026-07-29T11:35:50.590366129Z Wrote inspect data to must-gather.
2026-07-29T11:35:50.701039722Z INFO: 'previous-logs' folder not found on ip-10-0-135-214.ec2.internal, skipping...
2026-07-29T11:35:50.751943818Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-29T11:35:50.824568027Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-29T11:35:50.884189732Z Wrote inspect data to must-gather.
2026-07-29T11:35:50.966401701Z INFO: 'previous-logs' folder not found on ip-10-0-138-157.ec2.internal, skipping...
2026-07-29T11:35:50.966436298Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-29T11:35:50.966469408Z INFO: Machine Config Daemon termination log collection complete.
2026-07-29T11:35:50.998806565Z Wrote inspect data to must-gather.
2026-07-29T11:35:51.035925326Z INFO: Getting status/config from prometheus-k8s-0
2026-07-29T11:35:51.148906922Z Wrote inspect data to must-gather.
2026-07-29T11:35:51.150143111Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-29T11:35:51.391675774Z Wrote inspect data to must-gather.
2026-07-29T11:35:51.400972104Z Wrote inspect data to must-gather.
2026-07-29T11:35:51.451114751Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-29T11:35:51.603131657Z Wrote inspect data to must-gather.
2026-07-29T11:35:51.681336277Z INFO: INTERCONNECT MODE
2026-07-29T11:35:51.681381780Z INFO: Gathering ovn-kubernetes DBs
2026-07-29T11:35:51.693912386Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-29T11:35:51.852268507Z Wrote inspect data to must-gather.
2026-07-29T11:35:51.873463127Z INFO: Gathering OVN_Northbound from ovnkube-node-7kjb8...
2026-07-29T11:35:51.873771508Z INFO: Gathering OVN_Northbound from ovnkube-node-ddkml...
2026-07-29T11:35:51.874021928Z INFO: Gathering OVN_Northbound from ovnkube-node-jv7kt...
2026-07-29T11:35:51.874293901Z INFO: Gathering OVN_Southbound from ovnkube-node-7kjb8...
2026-07-29T11:35:51.874572901Z INFO: Gathering OVN_Southbound from ovnkube-node-ddkml...
2026-07-29T11:35:51.874854142Z INFO: Gathering OVN_Southbound from ovnkube-node-jv7kt...
2026-07-29T11:35:51.916723403Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T11:35:51.928653599Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-29T11:35:52.030435957Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-29T11:35:52.188072429Z tar: Removing leading `/' from member names
2026-07-29T11:35:52.202477643Z tar: Removing leading `/' from member names
2026-07-29T11:35:52.210207031Z tar: Removing leading `/' from member names
2026-07-29T11:35:52.245637122Z Wrote inspect data to must-gather.
2026-07-29T11:35:52.252981177Z tar: Removing leading `/' from member names
2026-07-29T11:35:52.273347762Z tar: Removing leading `/' from member names
2026-07-29T11:35:52.295093603Z tar: Removing leading `/' from member names
2026-07-29T11:35:52.325734562Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:52.414514625Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-29T11:35:52.491529051Z Wrote inspect data to must-gather.
2026-07-29T11:35:52.585703135Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:52.588129996Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-29T11:35:52.743220801Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T11:35:52.765101208Z INFO: Waiting for network log collection to complete ...
2026-07-29T11:35:52.765213466Z INFO: Waiting for ovnk database copies to complete ...
2026-07-29T11:35:52.765741118Z INFO: Copying ovnk databases complete.
2026-07-29T11:35:52.767324033Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-29T11:35:52.769712748Z ovnk_database_store/
2026-07-29T11:35:52.769764717Z ovnk_database_store/ovnkube-node-ddkml_sbdb
2026-07-29T11:35:52.789259163Z ovnk_database_store/ovnkube-node-7kjb8_nbdb
2026-07-29T11:35:52.803636709Z ovnk_database_store/ovnkube-node-7kjb8_sbdb
2026-07-29T11:35:52.817848420Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:52.826135650Z ovnk_database_store/ovnkube-node-jv7kt_nbdb
2026-07-29T11:35:52.832769674Z ovnk_database_store/ovnkube-node-ddkml_nbdb
2026-07-29T11:35:52.840540651Z ovnk_database_store/ovnkube-node-jv7kt_sbdb
2026-07-29T11:35:52.854480236Z Wrote inspect data to must-gather.
2026-07-29T11:35:52.860548917Z INFO: Network log collection complete.
2026-07-29T11:35:52.871478455Z INFO: Getting status from alertmanager-main-0
2026-07-29T11:35:52.946433132Z Gathering data for ns/openshift-network-console...
2026-07-29T11:35:53.063339280Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:53.066780979Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-29T11:35:53.174541045Z Wrote inspect data to must-gather.
2026-07-29T11:35:53.267320740Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:53.393069357Z Gathering data for ns/openshift-console-operator...
2026-07-29T11:35:53.438810982Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-29T11:35:53.441881156Z INFO: Fallback to identify the container image from release info
2026-07-29T11:35:53.491053695Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:53.494656261Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-29T11:35:53.700759293Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:53.773023253Z Gathering data for ns/openshift-console...
2026-07-29T11:35:53.892856376Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:53.896310726Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-29T11:35:54.045615736Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:54.170452406Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-29T11:35:54.255188141Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:35:54.274957412Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-29T11:35:54.305047950Z daemonset.apps/perf-node-gather-daemonset created
2026-07-29T11:35:54.444015789Z Waiting for performance profile collector pods to become ready: 1
2026-07-29T11:35:54.837774662Z Gathering data for ns/openshift-dns-operator...
2026-07-29T11:35:54.962674753Z Done executing Istio gather script
2026-07-29T11:35:55.118072102Z Gathering data for ns/openshift-dns...
2026-07-29T11:35:55.426325081Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-29T11:35:55.557763637Z Waiting for performance profile collector pods to become ready: 2
2026-07-29T11:35:55.685417679Z Gathering data for ns/openshift-image-registry...
2026-07-29T11:35:56.115281935Z Gathering data for ns/openshift-ingress-operator...
2026-07-29T11:35:56.418097550Z Gathering data for ns/openshift-ingress...
2026-07-29T11:35:56.677082961Z Waiting for performance profile collector pods to become ready: 3
2026-07-29T11:35:56.735051115Z Gathering data for ns/openshift-ingress-canary...
2026-07-29T11:35:57.088199914Z Gathering data for ns/openshift-insights...
2026-07-29T11:35:57.789111482Z Waiting for performance profile collector pods to become ready: 4
2026-07-29T11:35:58.873680989Z Gathering data for ns/openshift-jobset-operator...
2026-07-29T11:35:58.893167681Z Waiting for performance profile collector pods to become ready: 5
2026-07-29T11:35:59.624596975Z Gathering data for ns/openshift-config...
2026-07-29T11:35:59.868626125Z Gathering data for ns/openshift-config-managed...
2026-07-29T11:36:00.001622716Z Waiting for performance profile collector pods to become ready: 6
2026-07-29T11:36:00.334548052Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-29T11:36:00.430647667Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-29T11:36:00.599375661Z Gathering data for ns/openshift-kube-apiserver...
2026-07-29T11:36:00.895780969Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-29T11:36:01.116370174Z Waiting for performance profile collector pods to become ready: 7
2026-07-29T11:36:01.210861303Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-29T11:36:01.477621126Z Gathering data for ns/openshift-kube-scheduler...
2026-07-29T11:36:01.764364451Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-29T11:36:02.036223053Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-29T11:36:02.227112742Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-29T11:36:02.359428730Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-29T11:36:02.489576937Z Collecting performance related data for node ip-10-0-135-214.ec2.internal
2026-07-29T11:36:02.496720361Z Collecting performance related data for node ip-10-0-138-157.ec2.internal
2026-07-29T11:36:02.504318162Z Collecting performance related data for node ip-10-0-131-5.ec2.internal
2026-07-29T11:36:02.702064857Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-29T11:36:03.257554429Z Gathering data for ns/openshift-multus...
2026-07-29T11:36:04.710301294Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-29T11:36:05.435101141Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-29T11:36:05.954523661Z Gathering data for ns/openshift-host-network...
2026-07-29T11:36:06.236022689Z Gathering data for ns/openshift-network-diagnostics...
2026-07-29T11:36:06.760931990Z Gathering data for ns/openshift-network-node-identity...
2026-07-29T11:36:07.154033700Z Gathering data for ns/openshift-network-operator...
2026-07-29T11:36:07.509380994Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-29T11:36:07.768379346Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-29T11:36:08.236801493Z Gathering data for ns/openshift-apiserver-operator...
2026-07-29T11:36:08.521831294Z Gathering data for ns/openshift-apiserver...
2026-07-29T11:36:08.888400365Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-29T11:36:09.149828577Z Gathering data for ns/openshift-controller-manager...
2026-07-29T11:36:09.414208080Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-29T11:36:10.051571137Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-29T11:36:10.362687244Z Gathering data for ns/openshift-service-ca-operator...
2026-07-29T11:36:10.443882699Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-29T11:36:10.660398728Z Gathering data for ns/openshift-service-ca...
2026-07-29T11:36:10.817279665Z Collecting kubelet logs for node ip-10-0-135-214.ec2.internal
2026-07-29T11:36:10.819622205Z Collecting kubelet logs for node ip-10-0-138-157.ec2.internal
2026-07-29T11:36:10.821569059Z Collecting kubelet logs for node ip-10-0-131-5.ec2.internal
2026-07-29T11:36:11.061688362Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-29T11:36:11.161025770Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vblz7 namespace
2026-07-29T11:36:11.165547387Z INFO: Node performance data collection complete.
2026-07-29T11:36:11.941408430Z Wrote inspect data to must-gather.
2026-07-29T11:36:11.941455577Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T11:36:11.941455577Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-29T11:36:12.065812015Z error: the server doesn't have a resource type "clusters"
2026-07-29T11:36:12.203070712Z Caches written to disk