2026-07-31T10:09:00.272554234Z [disk usage checker] Started
2026-07-31T10:09:00.275844972Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-31T10:09:00.463940216Z Gathering data for ns/openshift-cluster-version...
2026-07-31T10:09:00.588865335Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T10:09:00.671689555Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T10:09:00.747693354Z Gathering data for ns/default...
2026-07-31T10:09:01.036255571Z Gathering data for ns/openshift...
2026-07-31T10:09:01.430900389Z Gathering data for ns/kube-system...
2026-07-31T10:09:02.151018878Z Gathering data for ns/openshift-etcd...
2026-07-31T10:09:02.444787126Z Wrote inspect data to must-gather.
2026-07-31T10:09:02.444821441Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T10:09:02.444821441Z     namespaces "assisted-installer" not found
2026-07-31T10:09:02.767508020Z Waiting on subprocesses to finish execution.
2026-07-31T10:09:02.775341180Z INFO: Gathering HAProxy config files
2026-07-31T10:09:02.777527949Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-31T10:09:02.781522776Z Executing Istio gather script
2026-07-31T10:09:02.796332909Z INFO: Waiting for node performance related collection to complete ...
2026-07-31T10:09:02.800771037Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-31T10:09:02.800925839Z INFO: Collecting host service logs for crio
2026-07-31T10:09:02.801277528Z INFO: Collecting host service logs for kubelet
2026-07-31T10:09:02.801569679Z INFO: Collecting host service logs for rpm-ostreed
2026-07-31T10:09:02.801862056Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-31T10:09:02.802179336Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-31T10:09:02.802470309Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-31T10:09:02.802768410Z INFO: Collecting host service logs for NetworkManager
2026-07-31T10:09:02.803069383Z INFO: Collecting host service logs for openvswitch
2026-07-31T10:09:02.803349732Z INFO: Collecting host service logs for ovs-configuration
2026-07-31T10:09:02.803639658Z INFO: Collecting host service logs for ovsdb-server
2026-07-31T10:09:02.803926616Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-31T10:09:02.804286903Z INFO: Waiting for worker host service log collection to complete ...
2026-07-31T10:09:02.806998113Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-31T10:09:03.350914356Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-31T10:09:03.435966132Z error: the server doesn't have a resource type "performanceprofile"
2026-07-31T10:09:03.446661714Z INFO: "metallb-operator" not detected. Skipping.
2026-07-31T10:09:03.509370685Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-31T10:09:03.604064037Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-31T10:09:03.639659969Z No resources found in openshift-etcd namespace.
2026-07-31T10:09:04.021153933Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-31T10:09:04.021238907Z INFO: on-disk MachineConfig config collection complete.
2026-07-31T10:09:04.170070148Z ERROR: No running kube-apiserver pods found
2026-07-31T10:09:04.207146842Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-31T10:09:04.209272822Z INFO: Collecting Insights Archives from insights-operator-77696685b-vmsvh
2026-07-31T10:09:04.209272822Z insights-runtime-extractor-75zt9
2026-07-31T10:09:04.209272822Z insights-runtime-extractor-9lrsj
2026-07-31T10:09:04.209272822Z insights-runtime-extractor-pr4md
2026-07-31T10:09:04.227233430Z INFO: Worker host service log collection to complete.
2026-07-31T10:09:04.268943420Z No resources found
2026-07-31T10:09:04.278844882Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-31T10:09:04.413292769Z Wrote inspect data to must-gather.
2026-07-31T10:09:04.502862007Z Wrote inspect data to must-gather.
2026-07-31T10:09:04.585424440Z Gathering data for ns/openshift-monitoring...
2026-07-31T10:09:04.600918259Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-31T10:09:04.900009766Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-31T10:09:04.900009766Z See 'oc rsync -h' for help and examples
2026-07-31T10:09:04.910398130Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T10:09:05.083805343Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-31T10:09:05.091078675Z Wrote inspect data to must-gather.
2026-07-31T10:09:05.119090321Z No resources found
2026-07-31T10:09:05.297466678Z tar: Removing leading `/' from member names
2026-07-31T10:09:05.303897748Z Wrote inspect data to must-gather.
2026-07-31T10:09:05.315036411Z INFO: HAProxy config collection complete.
2026-07-31T10:09:05.341827958Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-31T10:09:05.350630274Z Wrote inspect data to must-gather.
2026-07-31T10:09:05.351750158Z INFO: 'previous-logs' folder not found on ip-10-0-128-173.ec2.internal, skipping...
2026-07-31T10:09:05.380463957Z error: the server doesn't have a resource type "machineconfigs"
2026-07-31T10:09:05.396370910Z Wrote inspect data to must-gather.
2026-07-31T10:09:05.449855480Z Wrote inspect data to must-gather.
2026-07-31T10:09:05.481825793Z INFO: Getting rules from prometheus-k8s-0
2026-07-31T10:09:05.584679692Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-31T10:09:05.698064983Z INFO: 'previous-logs' folder not found on ip-10-0-134-200.ec2.internal, skipping...
2026-07-31T10:09:05.701377060Z Wrote inspect data to must-gather.
2026-07-31T10:09:05.858437610Z Wrote inspect data to must-gather.
2026-07-31T10:09:05.956431163Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-31T10:09:05.979877076Z INFO: Getting status/config from prometheus-k8s-0
2026-07-31T10:09:06.023886614Z Wrote inspect data to must-gather.
2026-07-31T10:09:06.027039847Z INFO: 'previous-logs' folder not found on ip-10-0-140-21.ec2.internal, skipping...
2026-07-31T10:09:06.027068336Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-31T10:09:06.027105773Z INFO: Machine Config Daemon termination log collection complete.
2026-07-31T10:09:06.062478045Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-31T10:09:06.246916158Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T10:09:06.311397195Z Wrote inspect data to must-gather.
2026-07-31T10:09:06.330286287Z Wrote inspect data to must-gather.
2026-07-31T10:09:06.429614379Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-31T10:09:06.549134957Z Wrote inspect data to must-gather.
2026-07-31T10:09:06.637058756Z INFO: INTERCONNECT MODE
2026-07-31T10:09:06.637171048Z INFO: Gathering ovn-kubernetes DBs
2026-07-31T10:09:06.664061270Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-31T10:09:06.761261936Z W0731 10:09:06.761214    1287 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-31T10:09:06.768376394Z Wrote inspect data to must-gather.
2026-07-31T10:09:06.817443008Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-31T10:09:06.819049399Z INFO: Gathering OVN_Northbound from ovnkube-node-4rqcj...
2026-07-31T10:09:06.819289047Z INFO: Gathering OVN_Northbound from ovnkube-node-5c777...
2026-07-31T10:09:06.819530582Z INFO: Gathering OVN_Northbound from ovnkube-node-q85s2...
2026-07-31T10:09:06.820024764Z INFO: Gathering OVN_Southbound from ovnkube-node-4rqcj...
2026-07-31T10:09:06.820306383Z INFO: Gathering OVN_Southbound from ovnkube-node-5c777...
2026-07-31T10:09:06.820593587Z INFO: Gathering OVN_Southbound from ovnkube-node-q85s2...
2026-07-31T10:09:06.997536822Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T10:09:07.022510786Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-31T10:09:07.116576673Z tar: Removing leading `/' from member names
2026-07-31T10:09:07.155266572Z tar: Removing leading `/' from member names
2026-07-31T10:09:07.160824371Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:07.198107567Z tar: Removing leading `/' from member names
2026-07-31T10:09:07.198993519Z tar: Removing leading `/' from member names
2026-07-31T10:09:07.202161189Z tar: Removing leading `/' from member names
2026-07-31T10:09:07.209265652Z tar: Removing leading `/' from member names
2026-07-31T10:09:07.246521342Z Wrote inspect data to must-gather.
2026-07-31T10:09:07.334057406Z Gathering data for ns/openshift-network-console...
2026-07-31T10:09:07.363205052Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-31T10:09:07.421629963Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:07.428035866Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-31T10:09:07.584678101Z Wrote inspect data to must-gather.
2026-07-31T10:09:07.671256755Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:07.692545113Z INFO: Waiting for network log collection to complete ...
2026-07-31T10:09:07.692659723Z INFO: Waiting for ovnk database copies to complete ...
2026-07-31T10:09:07.693179171Z INFO: Copying ovnk databases complete.
2026-07-31T10:09:07.695039596Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-31T10:09:07.698747778Z ovnk_database_store/
2026-07-31T10:09:07.698798681Z ovnk_database_store/ovnkube-node-5c777_nbdb
2026-07-31T10:09:07.709161636Z ovnk_database_store/ovnkube-node-5c777_sbdb
2026-07-31T10:09:07.729887408Z ovnk_database_store/ovnkube-node-4rqcj_sbdb
2026-07-31T10:09:07.749274906Z ovnk_database_store/ovnkube-node-4rqcj_nbdb
2026-07-31T10:09:07.755083508Z ovnk_database_store/ovnkube-node-q85s2_nbdb
2026-07-31T10:09:07.763079101Z ovnk_database_store/ovnkube-node-q85s2_sbdb
2026-07-31T10:09:07.783129483Z INFO: Network log collection complete.
2026-07-31T10:09:07.826432325Z INFO: Getting status from alertmanager-main-0
2026-07-31T10:09:07.910756141Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:07.913192377Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-31T10:09:07.916693963Z Wrote inspect data to must-gather.
2026-07-31T10:09:07.935818512Z Gathering data for ns/openshift-console-operator...
2026-07-31T10:09:08.106316504Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:08.184595682Z Wrote inspect data to must-gather.
2026-07-31T10:09:08.293663332Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:08.295928001Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-31T10:09:08.367220671Z Gathering data for ns/openshift-console...
2026-07-31T10:09:08.436852278Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-31T10:09:08.439491766Z INFO: Fallback to identify the container image from release info
2026-07-31T10:09:08.518138000Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:08.690128218Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:08.692169536Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-31T10:09:08.853706142Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:08.879713764Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-31T10:09:09.028296726Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-07-31T10:09:09.048375937Z Wrote inspect data to /must-gather/istio.
2026-07-31T10:09:09.153308953Z daemonset.apps/perf-node-gather-daemonset created
2026-07-31T10:09:09.290294911Z Waiting for performance profile collector pods to become ready: 1
2026-07-31T10:09:09.452082672Z Gathering data for ns/openshift-dns-operator...
2026-07-31T10:09:09.766740612Z Done executing Istio gather script
2026-07-31T10:09:09.780071253Z Gathering data for ns/openshift-dns...
2026-07-31T10:09:10.347679456Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-31T10:09:10.353236632Z Gathering data for ns/openshift-image-registry...
2026-07-31T10:09:10.409241428Z Waiting for performance profile collector pods to become ready: 2
2026-07-31T10:09:10.804392018Z Gathering data for ns/openshift-ingress-operator...
2026-07-31T10:09:11.106131784Z Gathering data for ns/openshift-ingress...
2026-07-31T10:09:11.532667108Z Gathering data for ns/openshift-ingress-canary...
2026-07-31T10:09:11.534148982Z Waiting for performance profile collector pods to become ready: 3
2026-07-31T10:09:11.897986386Z Gathering data for ns/openshift-insights...
2026-07-31T10:09:12.640633851Z Waiting for performance profile collector pods to become ready: 4
2026-07-31T10:09:13.661613689Z Gathering data for ns/openshift-jobset-operator...
2026-07-31T10:09:13.747467734Z Waiting for performance profile collector pods to become ready: 5
2026-07-31T10:09:14.437070651Z Gathering data for ns/openshift-config...
2026-07-31T10:09:14.768633416Z Gathering data for ns/openshift-config-managed...
2026-07-31T10:09:14.871598675Z Waiting for performance profile collector pods to become ready: 6
2026-07-31T10:09:15.194759884Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-31T10:09:15.352534950Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-31T10:09:15.505810006Z Gathering data for ns/openshift-kube-apiserver...
2026-07-31T10:09:15.759471479Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-31T10:09:15.982804334Z Waiting for performance profile collector pods to become ready: 7
2026-07-31T10:09:16.065523947Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-31T10:09:16.368218331Z Gathering data for ns/openshift-kube-scheduler...
2026-07-31T10:09:16.638318408Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-31T10:09:16.927515346Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-31T10:09:17.097089608Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-31T10:09:17.297915826Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-31T10:09:17.346729181Z Collecting performance related data for node ip-10-0-140-21.ec2.internal
2026-07-31T10:09:17.357886033Z Collecting performance related data for node ip-10-0-134-200.ec2.internal
2026-07-31T10:09:17.371458077Z Collecting performance related data for node ip-10-0-128-173.ec2.internal
2026-07-31T10:09:17.695108718Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-31T10:09:18.464065357Z Gathering data for ns/openshift-multus...
2026-07-31T10:09:19.995342191Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-31T10:09:20.358622082Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-31T10:09:21.258706291Z Gathering data for ns/openshift-host-network...
2026-07-31T10:09:21.585996679Z Gathering data for ns/openshift-network-diagnostics...
2026-07-31T10:09:22.192308225Z Gathering data for ns/openshift-network-node-identity...
2026-07-31T10:09:22.680913708Z Gathering data for ns/openshift-network-operator...
2026-07-31T10:09:23.152298741Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-31T10:09:23.425210493Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-31T10:09:23.907615290Z Gathering data for ns/openshift-apiserver-operator...
2026-07-31T10:09:24.249857540Z Gathering data for ns/openshift-apiserver...
2026-07-31T10:09:24.665697696Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-31T10:09:24.959228937Z Gathering data for ns/openshift-controller-manager...
2026-07-31T10:09:25.194794485Z Collecting kubelet logs for node ip-10-0-140-21.ec2.internal
2026-07-31T10:09:25.197159762Z Collecting kubelet logs for node ip-10-0-134-200.ec2.internal
2026-07-31T10:09:25.199370544Z Collecting kubelet logs for node ip-10-0-128-173.ec2.internal
2026-07-31T10:09:25.269401085Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-31T10:09:25.372011816Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-31T10:09:25.524942436Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-lxpdl namespace
2026-07-31T10:09:25.531161150Z INFO: Node performance data collection complete.
2026-07-31T10:09:25.958169304Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-31T10:09:26.254781160Z Gathering data for ns/openshift-service-ca-operator...
2026-07-31T10:09:26.575438739Z Gathering data for ns/openshift-service-ca...
2026-07-31T10:09:27.005125315Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-31T10:09:27.903066972Z Wrote inspect data to must-gather.
2026-07-31T10:09:27.903107979Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T10:09:27.903107979Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-31T10:09:28.022833406Z error: the server doesn't have a resource type "clusters"
2026-07-31T10:09:28.155930214Z Caches written to disk